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FOREWORD 


This  report  represents  one  phase  of  an  effort  directed  at  the  use  of  per¬ 
formance  criteria  as  elements  in  flight  control  system  optimization  studies. 
The  research  reported  was  sponsored  by  the  Flight  Control  Lahoratoiy  of  the 
Aeronautical  Systems  Division  under  Project  No.  8219.  It  was  conducted  at 
Systems  Technology ^  Inc.^  under  Contract  No.  AF  53(6l  6) -784l  .  A  companion 
report  (Ref.  1  )  dealing  with  performance  criteria  for  deterministic  inputs 
has  already  been  published  under  this  contract.  The  ASD  project  engineers 
were  Mr.  R.  0.  Anderson  and  Lt.  L.  Schwartz  of  the  Flight  Control  Laboratory. 
The  principal  contributors  to  the  report  are  listed  as  authors. 

The  authors  wish  to  express  their  thanks  to  Messrs.  D.  T.  McRuer  and 
Dunstan  Graham^  principal  investigators ^  who  planned  the  general  approach 
followed  in  both  reports  and  contributed  many  details.  Thanks  are  also  due 
to  Lt.  L.  Schwartz  for  his  thorough  check  of  and  valuable  comments  on  the 
report,  and  to  Mr.  A.  V.  Phatak  for  contributing  many  of  the  calculations. 
Acknowledgment  is  gratefully  made  to  Messrs.  J.  Taira  and  R.  N.  Nye  and 
Misses  N.  Crawford  and  D.  Lewis  for  their  careful  work  in  preparing  the 
report. 


ABSTRACT 


A  critical  siirvey  and  assessment  has  been  made  of  performance  measures 
and  associated  criteria  for  linear  constant-coefficient  systems  v/ith  random 
inputs.  The  suitability  of  each  criterion  for  flight  control  system 
optimization  using  "pencil  and  paper"  methods  has  been  investigated  by  con¬ 
sidering  its  validity,  selectivity,  and  ease  of  application.  Simplifications 
are  introduced  by  substituting  for  actual  flight  control  systems  lower  order 
equivalent  systems  having  similar  dynamics,  and  by  replacing  certain  random 
inputs  and  criteria  with  transient  analogs  and  "con^jatible"  deterministic 
criteria.  The  latter  sin5)lif ication  enables  the  calculation  of  random  input 
performance  measures  to  be  replaced  by  calculations  involving  more  easily 
visualized  deterministic  quantities . 

It  is  shown  that,  for  stationary  inputs,  a  wide  variety  of  criteria 
reduce  to  minimum  mean  square  error.  This  criterion  is  easy  to  use  and 
has  a  compatible  deterministic  form,  but  yields  lightly  damped  systems  and 
is  unsel active  (i.e.,  the  mean  square  error  of  a  wide  variety  of  off-optimum 
systems  is  little  higher  than  that  of  the  optimum) .  Hov/ever,  no  criterion 
was  found  that  had  the  advantages  of  minimum  mean  square  error  without 
comparable  or  worse  disadvantages.  For  certain  nonstationary  problems, 
the  probabilistic  square  error  criterion  appears  promising.  N\mierous 
improved  techniques  for  evaluating  criteria  and  several  exan^les  are 
presented.  It  is  concluded  that  none  of  the  criteria  that  have  been  pro¬ 
posed  to  date  are  suitable  as  sole  criteria  for  flight  control  system 
optimization. 
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NOMENCIATURE 


a 

A  parameter  used  to  obtain  the  Wiener  system 

a 

Input  power  spectrum  break  frequency 

A  random  variable  with  a  known  Gaussian  joint  probability 
density  function 

Loss  function  coefficient 

a(t) 

A  general  stationary  random  signal 

A 

See  Eq  207 

^Q} 

Denominator  coefficients  of  a  general  second-order  system 
transfer  function  (see  Eq  96) 

A(s) 

See  Eq  88 

A  =  A(jm) 

See  Eq  201 

b(t) 

A  general  stationary  random  signal 

B-i 

Numerator  coefficients  of  a  general  second-order  system 
transfer  function  (see  Eq  96) 

B(s) 

See  Eq  (88) 

B  =  B(jcn) 

See  Eq  201 

c 

System  output  c(t) 

Cd 

Desired  system  output  c^(t) 

Output  due  to  system  having  weighting  function  g(t) 

A  constant 

Transient  analog  (Chapter  IV)  of  c 

C(s) 

=  jClc  (t)  ] 

Output  of  the  Wiener  system 

e 

=  2,712  . 

erfc  (x) 

The  complementary  error  function  =  1  -  erf  (x) 

E 

Expected  value 

E(s) 

Ei(s) 
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Et(T) 

Generalized  error  fimction  for  deterministic  inputs 

EMSvf 

Frequency-weighted  mean  square  error 

f 

Frequency^  =  a)/2jt 

f,  f(t) 

Signal  or  command  portion  of  the  input 

f(r) 

Nonlinear  gain  function  operating  upon  input 

f(€) 

Benedict  and  Sondhi’s  loss  function 

fd(t) 

See  Glover *s  criterion  and  Table  I 

fk(t) 

A  known  deterministic  function  of  time 

Fourier  transform  operator 

g(t) 

(i)  A  general  weighting  function 

(ii)  A  deterministic  signal  representable  as  a  polynomial  in  t 

G 

Open- loop  transfer  function 

G.E.F. 

Generalized  error  function 

h,  h(t) 

General  transfer  function 

h(x) 

See  Eq  1 1 4 

h;i(t) 

Weighting  function  of  the  Wiener  system 

H  =  H(s) 

A  transfer  function 

Ha(s),  H^3(s) 

See  Eq  1 67 

H^(s) 

A  transfer  function 

Optimum  frequency-weighted  mean  square  error  system 

Hp(s) 

Transfer  function  of  the  Phillips  system 

Hw(s) 

Transfer  function  of  the  Wiener  system 

Hx(s) 

See  Eq  20^ 

Ie2 

^00 

Integrated  error  squared,  e^dt 

ITAE 

Integrated  time  moment  of  absolute  error,  t|e|dt 

ITE2 

1*00 

Integrated  time  moment  of  error  sqxxared,  te^dt 
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j 

J(s) 

k(t) 

ki(t) 

,2 

K 

KmKe 

L 

L(s) 

JZ 

m(t) 

Md 

Mr 

WIFE 

msE 

n 

n,  n(t) 

nt(t) 

N 

% 


No 


N 


€0 


=  Apr 

See  Eq  89 

Optimum  transfer  function  defined  ty  Eq  106 

An  "ideal"  transfer  fimction  defined  by  Eq  67 

Constant  defining  optimum  weighting  function  for  minimum 
probabilistic  square  error 

See  Eq  1 20 

Input  amplitude  descriptor  (see  Fig.  17) 

Product  of  controller  gains 

(i)  A  specified  level  of  output  or  effort 

(ii)  A  loss  function^  L(€) 

See  Eq  89 

Laplace  transform  operator 
A  stationary  random  signal  with  zero  mean 
A  deterministic  input  performance  measure 
A  random  input  performance  measure 
Mean  weighted  fiuiction  of  error^  see  Eq  "jk 
Mean  weighted  square  error^  see  Eq  75 
An  exponent 

Noise  or  unwanted  component  of  the  input 
Transient  analog  of  random  noise 

(i)  Largest  exponent  in  Kaufman’s  performance  measure 

(ii)  Noise  aii:5)litude 

Average  number  of  maxima  per  second  of  a  stationary  random 
qioantity 

Average  number  of  zero  crossings  per  second  of  a  stationary 
random  quantity 

Average  number  of  zero  crossings  per  second  of  effort 
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p 

p(a) 

p(a,b) 

p(a/b) 

Pi 

P2 

P(t) 


P.E. 


r 


rt(t) 


Pn 

R(s) 

Rt(s) 

%ab 


s,  s(t) 
s^(t) 


Probability  criterion  (page  55) 

Probability  density  function  of  a 

Joint  probability  density  function  of  two  quantities,  a  and  b 
p(a,b)  , . , . 

=  p(-b)  •’  conditional  probability  density  function  of  a,  given  b 

Zeros  of  L(s) 

Second  probability  density  function 

Weighting  function  for  end- sigma  performance  measure 


Glover's  performance  measure, 


lim  1 


dt 


An  input 

A  sequence  of  past  input  values^  r-j  ^  ••• 

A  sampled  value  of  the  input 

Transient  analog  of  a  random  signal  r(t) 

A  deterministic  input 
A  stationary  random  input 


lljn  J_  f-^ 
T-00  2tJ_^  ‘ 


a(t)b(t  +  T)dt,  crosscorrelation  between  two 
stationary  random  signals,  a(t)  and  b(t) 


Special  autocorrelation  functions  defined  in  Eq  71 

Laplace  transform  of  input 

=  /?[rt(t)] 

Weighted  crosscorrelation  or  autocorrelation  (a  =  b)  function 
see  Eq  116  ^ 

laplace  transform  con^jlex  "variable,  a  +  Jcu 

Signal  or  command  portion  of  the  input 

Transient  analog  of  random  signal 
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S  =  S(s) 

t 

=  X*!  (ja))X'j  (-jco)  energy  spectral  density 

Time 

■td 

Delay  time 

^1 

Time  of  application  of  a  step  input 

T 

(i)  A  fixed  time 

(ii)  Sampling  period 

( iii)  Observation  time 

u 

A  dummy  time  variable 

u(t) 

A  step  function  commencing  at  t  =  0 

V 

General  signal  break  frequency 

w(t) 

A  time-weighting  function 

W 

=  W(a))  (see  below) 

w(t) 

A  time -weighting  function 

W(cb) 

Frequency  weighting  fmction 

Wa(ai) 

See  Eq  224 

W-b(ai) 

See  Eq  225 

X 

A  dummy  time  variable 

xi  (t),  X2(t) 

Functions  of  time 

Xi(s) 

=5'[*i  (t)] 

X2(s) 

y 

A  dummy  time  variable 

z 

See  Eq  59 

a 

See  Eq  21 1 

“1 

A  system  adjustable  parameter 

See  Eq  39 

a-|  ,  02 

Weighting  parameters 
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See  Eq  211 


7^  A  Lagrange  multiplier 

r(x)  Gamma  function  of  x 
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(i)  A  constant  in  Glover *s  criterion  indicating  the  lowest 
accuracy  of  interest 

(ii)  Unit  impulse  function 

(i)  System  error  (difference  between  actual  and  desired  output) 

(ii)  Actuating  error  (difference  between  input  and  output) 

Transient  signals  used  to  generate  the  transient  analog  of  the 
cross  power  spectra  in  Chapter  IV 


Random  component  of  error 

Frequency- weighted  error  signal 

(t)  Transient  analog  of  random  signal 

e2(t)  Transient  analog  of  random  noise 

Systematic  error  at  time  tQ^  i.e.^  that  part  of  the  error  response 
due  to  the  deterministic  component  of  the  input 


n 

Til 

ea(^) 

K 

V 

^aa 

a2 

%b 


Damping  ratio 

Prediction  or  interpolation  time 
See  Eq  42 

Orthogonal  polynomial  in  r  of  degree  a  used  in  synthesizing 
Lubbock's  optimum  system 

Constant 

moment  of  impulsive  response  as  defined  in  Eq  65 
A  system  parameter 

Normalized  autocorrelation  function^  = 

aa 

Mean  square  value  of  some  quantity 


4 


Normalized  crosscorrelation  function^ 


^^ab(^) 

a  b 
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(i)  End-sigma  performance  measure 

^00 

Jo  V,  vi  ,  V2,  y-n 

(ii)  Probabilistic  square  error 

X  [e('t)J^(t)dt 

T 

A  dummy  time  variable 

CVJ 

b- 

Time  delays 

4>(cd) 

Power  spectral  density 

Input  power  spectrum,  =  4>i.p( jm) 

Error  power  spectrum 

A  general  cross- spectrum 

X 

N  _ 

Kaufman *s  performance  measure^ 

n=l  ^ 

A  function  of  present  and  past  input  samples 

^ir^  ^rr 

See  Eq  101 

CO 

Frequency^  imaginary  part  of  Laplace  transform  complex 
variable^  s  =  a  +  jco 

AAA/v^ 

1  1 

Time  average 

Ensemble  average 

Absolute  value 

[]^ 

Complex  conjugate  of  ^ 

[]" 

The  latio  of  the  left-hali'-plane  poles  and  zeros  of  the 
expression  in  brackets  (see  Eq  1  9) 

[]- 

The  ratio  of  the  right-half -plane  poles  and  zeros  of  the 
expression  in  brackets  (see  Eq  19) 

+ 

1 - 1 

1 - 1  •  li  III 

Expand  the  expression  in  brackets  in  partial  fractions  and  then 
keep  only  fractions  with  left-half -plane  poles  (see  Eq  1 9) 

Approximately  equals 

Identically  eqmls 
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INTRODUCTION 


A.  ARRANGEMENT  AND  CONTENTS  OF  THE  REPORT 

The  reader  who  wishes  to  obtain  a  more  complete  knowledge  of  random  input 
performance  criteria  than  can  be  gleaned  from  the  Abstract  and  Conclusions  alone, 
but  who  lacks  the  time  or  inclination  to  read  the  whole  report,  is  advised  to 
read  the  remainder  of  this  Introduction,  the  summaries  on  pp-  28j  and  65, 

and  all  of  the  concluding  chapter. 

This  report  presents  the  results  obtained  during  the  second  phase  of  a  general¬ 
ized  study  of  dynamic  performance  measures  for  automatic  flight  control  systems. 
Such  systems  are  subject  to  both  random  and  deterministic  inputs,  and  it  has  been 
found  convenient  to  present  the  results  of  the  study  in  two  parts.  Ref.  1  dealing 
with  deterministic  inputs  and  the  present  report  with  random  inputs-  This  intro¬ 
ductory  section  defines  some  important  terms  used  throughout  the  report,  and  sets 
out  the  viewpoint  from  which  the  various  performance  criteria  will  be  assessed- 
The  report  consists  principally  of  a  survey  and  critical  assessment  of  published 
performance  criteria.  The  number  and  diversity  of  the  criteria  examined  is  so 
great  that  it  would  take  many  pages  of  text  to  summarize  the  results  of  the  study 
in  such  a  fashion  that  something  is  said  about  each  criterion.  Table  I  has, 
therefore,  been  prepared;  it  lists  the  criteria,  states  the  applicable  input 
conditions,  and  briefly  summarizes  the  assessment  of  each  criterion  given  in  the 
main  text  of  the  report.  In  order  to  keep  the  size  of  Table  I  within  reasonable 
bounds,  the  criteria  definitions,  etc.,  are  terse;  fuller  explanations  will  be 
found  on  the  pages  indicated  in  the  table. 

Many  criteria  prove  to  be  equivalent  or  closely  related  to  other  criteria. 

These  relationships  are  compactly  illustrated  by  ” family  trees”  given  on  pp.  27, 

28,  and  63. 

Some  familiarity  is  presumed  on  the  part  of  the  reader  with  the  elements  of 
random  process  control  theory.  Definitions  of  such  standard  terms  as  "station¬ 
ary"  and  "Gaussian"  are  given  in  Ref.  2  and  elsewhere,  and  it  is  not  thought 
necessary  to  repeat  them  here.  It  is  also  desirable  (though  not  essential)  that 
the  reader  have  some  acqiiaintance  with  Wiener  and  Phillips-type  optimization 
because  only  a  brief  review  of  these  well-known  topics  is  given.  Unless  otherwise 
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stated,  it  may  te  assumed  that  random  inputs  referred  to  are  stationary  and  that 
the  systems  are  time -invariant  and  that  the  ergodic  hypothesis  is  applicable. 


B.  DEFINITIONS  OF  "PERFORMANCE  MEASURE,"  "PERFORMANCE  CRITERION, 
"LOSS  FUNCTION,"  AND  "EQUIVALENT  SYSTEM" 


Performance  Measure :  A  quantity  characterizing  some  aspect  of  dynamic  per¬ 
formance,  such  as  stability,  response  to  desired  inputs,  response  to  unwanted 
inputs,  accuracy,  etc. 

Performance  Criterion :  A  standard  or  reference  value  of  some  performance 
measure.  It  provides  a  basis  for  a  rule  or  test  by  which  some  aspect  of  dynamic 
performance  is  evaluated  in  forming  a  judgment  of  system  quality. 

Loss  Function:  A  function  only  of  the  controlled  quantity,  c(t) ,  or  of  the 
instantaneous  error,  €(t),  between  the  desired  and  actual  values  of  c(t).  Its 
magnitude  at  any  given  €(t)  | or  c(t)|  indicates  the  importance  attached  to  that 
e(t)  |or  c(t)| .  For  example,  in  Fig.  1  the  loss  function  is  appropriate  to  a 


Loss  Function,  L 


I 


LU) 


Figure  1 .  Typical  Loss  Function 

situation  where  all  errors  greater  than  [cq!  are  of  equal  importance,  while  errors 
of  smaller  magnitude  are  of  zero  importance.  Most  of  the  performance  measures 
discussed  in  this  report  are  time-averaged  loss  fimctions  of  error,  L(e)  (for 
example,  e^) . 


Equivalent  System:  A  simpler  system  which  has  closed-loop  dynamic  character¬ 
istics  approximately  equivalent  to  those  of  some  actual  system.  It  normally  has  a 
transfer  function  of  first  to  fourth  order  plus,  if  necessary,  time  delay  terms  to 
represent  high-frequency  leads  and  lags. 

The  general  procedure  by  which  equivalent  systems  are  derived  is  most  clearly 
illustrated  by  an  example  taken  from  Ref.  1  and  repeated  here  for  ease  of  reference. 
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TABLE  I 

SUMMARY  OF  RAKDOM  INPUT  PERFORMANCE  MEASURES 
For  Linear  Constant-CoefflGient  Systems 


MEASURE 


SEE 

IPAGE 


TYPE  OF  SYSTEM  INPUTS 


ASSOCIATED  CRITERION 


CRITERION  ASSESSMENT 


REMARKS 


Mean  square  error 


IT 


T  _ 

^  ”  T-*<D 


j-r\ 


stationary  random  desired  and  unwanted 
signals  (Ref.  3  and  4). 


■(t)dt 


Minimum  value.  System  form  may 

be  fixed  (Phi  Hips -type  optimiz¬ 
ation)  or  free  (Wiener  optimiza¬ 
tion)  .  For  Wiener  optimization; 
system  form  is  implicitly  deter¬ 
mined  by  form  of  input  and 
desired  response  spectra. 


gS  tends  to  be  unselective 

and  the  optimum  system  not 
well  damped.  If  input 
signals  are  stationary  and 
Gaussian;  the  Wiener  optimum 
linear  system  is  the  absolute 
optimum  of  all  filters,  linear 
or  nonlinear. 


Exceptionally  well  developed  theory  and  application  (Ref.  3,  5^ 

6  ;  7  ,  and  8 ) .  Primary  difficulty  in  flight  control  application 
to  find  adequate  expressions  for  desired  and  unwanted  signals. 
gP  can  be  expressed  analytically  in  terms  of  system  and  signal 
parameters;  hence,  the  effect  of  parameter  variations  on  ^  can  be 
assessed  in  a  straightforward  manner.  In  practice,  high-order 
systems  yield  complicated  expressions,  and  trial  and  error  is  used 
to  supplement  the  analytical  treatment  (Ref.  6). 


119 


Stationary  random  desired  signal  plus 
constraints  on  system. 


57 


Command  signal  composed  of  a  polynomial 
time  function  of  order  n,  plus  a  sta¬ 
tionary  random  signal  plus  an  unwanted 
input  which  is  a  stationary  random  signal 


Example : 

Minimum  ^  while 
T 

( torque  )^dt  =  const. 
T  00  2T  ^  ' 

(Ref.  9  and  10)  or  minimuir 
lighted  plant  input  power  with 
e2  <  constant  (Ref.  10  and  11 ) 
or  other  system  constraints. 

Minimiim  e2  [first  n  moments  of 

E(8)/R(s)  specl'<'ied] 

(Ref.  7  and  12). 


No  general  assessment  due  to 
wide  variety  of  possible  con¬ 
straints. 


Well-developed  theory  and  application  using  Lagrange  multiplier 
techniques  (Ref.  6,  9,  10,  and  ll). 


Even  simple  problems  require 

considerable  computational 
effort. 


Well-developed  theory  combining  error  coefficients  with  random 
input  system  theory  (Ref.  7  and  12). 


Mean  function  of  error 
lim 


26 


Stationary  Gaussian  random  desired  and 
unwanted  signals. 


Minimum  value.  System  form. 

(i)  free  or  (li)  fixed. 


Yields  minimum  system 

[(l)  Wiener  or  (ii)  Phillips]  . 


tie)  =  |e|"'  +  |e|”^  +  • 

need  not  be  an  integer. 


Ul 


Each  n  must  be  positive,  but 


f^T^ 


T-.-00  ^ 


f  (e)dt 


Stationary  random  desired  and  unwanted 
signals . 


Minimum  value.  System  form 
free. 


55 


Stationary  Gaussian  random  desired  and 
imwanted  signals. 


Minimum  value.  System  form 
(i)  free  or  (ii)  fixed. 


34  Gaussian  random  desired  (Ref.  1 4)  and 

unwanted  signals  plus  additional  signal 


Minimum  value.  System  form 
free. 


terms  of  the  form 


fk(t)'s  are  known  deterministic 
functions 

ajj^'s  are  random  variables  with  a 
known  Gaussian  joint  proba¬ 
bility  density  function 


A)  For  stationary  Gaussian 
processes,  yields  the 
Wiener  system. 

B)  If  signals  are  nonstation¬ 
ary  Gaussian,  the  optimum 
system  is  a  time-varying 
linear  system  which  also 
minimizes  e^. 

C)  For  non -Gauss Ian  (station¬ 
ary  oi'  nonstationary) 
signals,  the  general 
solution  is  not  known. 


Sherman'S  (Ref. 

15) 

1  loss 

function,  f(€),  is  defined 

f(o) 

= 

0 

null  property 

f(e) 

= 

f(-e) 

symmetry  property 

f(ei) 

> 

fCep) 

>  0  monotonic  property 

for 

> 

^2  > 

0 

The  determination  of  the  optimum  system  also  depends  on  some  prop¬ 
erties  of  the  input  distribution  (see  text). 


Yields  minimum  system 
[(i)  Wiener  or  (ii)  Phillips] 


The  absolute  optimum  system 
is  the  minimum  linear 
(possibly  time-varying)  system 
plus  a  bias  term  on  the  out¬ 
put. 


Chang's  (Ref.  10  )  loss  function  is  the  same  as  Shei^n’s  (Ref.  13) 

above,  although  it  may  be  asymmetric  if  in  addition  e  s  0. 

Brown  (Ref.  15)  also  considered  an  asymmetric  loss  function. 


The  loss  function  is  an  arbitrary  function  of  the  error.  If  it  is 
the  same  as  Sherman's  (Ref.  13)  loss  function  (see  above),  then 
the  output  bias  term  is  zero. 
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TABLE  I  (Continued) 

SUMMARY  OF  RAMDOM  INPUT  PERFORMANCE  MEASURES 
For  Linear  Constant-Coefficient  Systems 


MEASURE 

"See 

PAGE 

TYPE  OF  SYSTEM  INPUTS 

ASSOCIATED  CRITERION 

CRITERION  ASSESSMENT 

REMARKS 

Kaufman's  performance 
measure  (Ref.  l6) 

14  ,  TT  W 

Xim  1  1  2n  j 

55 

Stationary  random  desired  and  unwanted 
signals. 

Minimum  value  (Ref.  l6). 

System  order  specified. 

Yields  the  same  system  as  ^ 
for  Gaussian  inputs.  Very 
few  results  for  non-Gaussian 
inputs . 

Original  reason  for  this  criterion  was  to  handle  non-Gaussian 
signals.  Previous  work  (Ref.  l6)  considered  only  linear  systems 
with  inputs  having  probability  density  functions  symmetric  about 
the  mean.  In  some  cases,  Kaufman's  performance  measure  is  mini¬ 
mized  by  minimizing 

Averag 
exceedj 
of  Bom< 
level. 

3  number  of 
ances  per  sec( 
2  specified 

L 

/2o2 

^  (oj)  das 

Dnd 

1/2 

4i 

Stationary  Gaussian  random  inputs 
(Ref.  17). 

Minimum  value  of  output  or  error 
exceedances.  System  order 
specified. 

Examples  studied  to  date,  for 
both  output  and  error  exceed¬ 
ances,  show  that  the  criteria 
tend  to  select  very  heavily 
damped  systems. 

Originally  proposed  as  a  fatigue  measure  (output  exceedances) . 

p  =  Prob  1 1 e 1  <  L } 

L  Is  a  specified  toler¬ 
ance 

54 

Stationary  Gaussian  random  input,  plus 
deterministic  component. 

Maximum  value  (Ref.  6,  l8,  and 
19).  System  order  specified 
(Ref.  18). 

Not  generally  useful  as  form¬ 
ulated  in  Ref.  l8. 

Ref.  18  considered  the  case  when  the  error  has  two  components  such 
that  e[€^J^^  =  e^Q  +  0^  where  ego  is  the  systematic  error  at  time 

tQ  and  ci2  the  variance  of  the  stationary  random  component  of  the 
error.  Then,  if 

[^sol  ^^sO 

^bo  »  a,  and  =  L 

it  Is  claimed  that  maximizing  pis  the  same  as  minimizing  e[€^J  . 

This  is  strictly  true  only  if  GgQ  is  independent  of  the  free 
parameters.  For  this  case,  minimizing  maximizes  p  independently 

of  the  above  conditions.  If  ego  i®  independent  of  the  free 

parameters,  then  there  is  no  guarantee  that  the  above  conditions 
hold  as  parameters  are  varied. 
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TABLE  I  (Continued) 

SUMMARY  OP  RANDOM  INPUT  PERFORMANCE  MEASURES 
For  Linear  Constant- Coefficient  Systems 


MEASURE 

SEE 

PAGE 

TYPE  OP  SYSTEM  INPUTS 

ASSOCIATED  CRITERION 

CRITERION  ASSESSMENT 

REMARKS 

Time -weighted  mean 
square  error  (Ref.  20) 

T-m  ^  J_^v(t)62(t)dt 

w(t)  >  0  for  -00  <  t  <oo 

62 

Stationary  random  inputs. 

Minimum  value.  System  order 
free  (Ref.  20)  or  specified. 

Yields  the  same  optimum  system 
as  the  "eS  (wiener  or  Phillips) 
system. 

For  w(t)  statistically  independent  of  €(t)  (either  deterministic 
or  nondeterministic),  performance  measure  is  shown  to  be  equal  to 
(^)  (w) .  Ref.  20  considers  the  case  of  deterministic  w(t)  only. 

Glover’s  amplitude - 
weighted  error  (Ref.  21  ) 

T-co  S 

vdere  «(t)  =  ^ 

a  nondeterministic  func¬ 
tion, 

fjj(t)  is  desired  signal, 

6^  indicates  the  lowest 
accuracy  of  interest. 

75 

Stationary  random  inputs. 

Minimum  value .  System  order 
free  (Ref.  21  )  or  specified. 

Tends  to  be  very  unselectlve 
when  system  order  is  speci¬ 
fied. 

For  w(t)  not  statistically  independent  of  g(t)  and  for  Gaussian 
processes,  the  optimvmi  system  is  the  Wiener  system  multiplied  by  a 
constant  (which  is  a  function  of  the  input  and  system  parameters) 

(Ref.  21  ) .  Optimization  with  transfer  functions  of  specified  order 
(Phillips  method)  yields  systems  not  related  to  the  Phillips  system 
in  any  simple  manner. 

End  sigma 

66 

Combined  stationary  or  nonstationary 
random  and  deterministic  inputs. 

Minimum  value  (Ref.  22)and  2^)  • 

Valuable  for  nonstationary 
problems  where  precise  formu¬ 
lation  of  p(t)  is  possible. 

Proposed  as  an  all-encompassing  criterion  (Ref.  22).  The  integrand 
is  a  function  of  various  system  parameters,  V| ,  vg,  v^,  •••  and 

p(t),  the  probability  distribution  of  all  times  when  the  system 
output  is  utilized.  Published  examples  all  use  simplified 
criterion  given  below. 

9=  1  F{t)F{€(t),  t,  vi,  1 

Jo 

■*  Vnldt 

For  stationary  situations, 
reduces  to  simpler  criteria, 
or  Is  unsuitable. 

Probabilistic  square 
error 

r  p(t)e2(t)dt 

Jo  ^ 

66 

Combined  stationary  or  nonstationary 
random  and  deterministic  inputs. 

Minimum  value.  Maximum  system 
order  specified  either 
explicitly  or  implicitly 

Special  form  of  "end  sigma" 
criterion.  Assessment  as 
above . 

Complete  digital  computer  evaluation  procedures  using  Legendre 
polynomials  are  available  (Ref.  24,  25,  26,  and  27).  Analytic 
procedures  discussed  in  this  report. 
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TABLE  I  (Concluded) 

SUMMARY  OF  RANDOM  INPUT  PERFORMANCE  MEASURES 
For  Linear  Constant -Coef^ficient  Systems 


MEASURE 

SEE 

PAGE 

TYPE  OF  SYSTEM  INPUTS 

ASSOCIATED  CRITERION 

CRITERION  ASSESSMENT 

REMARKS 

Generalized  error 
function  (Ref.  39,  88) 

87 

t  -  T 

Stationary  random  signal  and  noise. 

-  c(t)]%t 

Minimum  value.  System  form 
free  or  speclflicd. 

When  system  order  is  free, 
yields  Wiener  system  for  the 
appropriate  t  (prediction  or 
interpolation) .  When  the 
system  order  is  specified 
(Phillips  system),  measure  is 
difficult  to  minimize  analyt¬ 
ically  since  the  expressions 
for  G.E.F.  contain  exponential 
terms. 

This  measure  has  a  compatible  form  for  deterministic  inputs 

E(t)  =  f  [s(t  -  t)  -  c(t)J^dt 

Jo 

In  Ref.  28,  £(7:)  is  called  ISDE  (integral  square  delayed  error) 
when  there  is  zero  noise. 

Frequency-weighted  mean 
square  error  (Ref.  32) 

EMSw  =  ^  *ee(a))W(a))d< 

J-oo 

105 

n 

Stationary  random  signal  and  noise. 

Minimum  value.  System  form 
free  or  specified. 

Free  order  optimum  systems  are 
related  to  Wiener  system. 

Useful  criterion  when  W(o) 
can  be  specified  so  that  it 
reflects  the  physical  require¬ 
ments  of  the  system  (see 
below) . 

W(a))  should  be  largest  for  frequency  regions  where  error  power  is 
undesirable.  W(aj)  must  not  tend  to  zero  as  m  tends  toward 
infinity  if  system  order  is  free. 

Mean  square  error  deriv¬ 
atives  (Ref.  55) 

lim  1  n  y  rd^e(t)l' 

112 

2 

dt 

1 

Stationary  random  signal  and  noise. 

Minimum  value.  System  order 
free  or  specified. 

Simple  forms  of  this  criterion 
are  studied  in  Chapter  V,  and 
it  is  shown  that  they  fail  to 
achieve  good  selectivity. 

This  measure  is  a  form  of  the  frequency-weighted  mean  square  error 
measure  {w(a))  is  then  a  numerator  polynomial  in  . 
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Consider  a  pitch  control  system  for  the  fighter  airplane  detailed  in  Appendix  C 
of  Ref.  1.  The  open-loop  transfer  function  for  the  pitch  loop  is 


G(s)  = 


-372  ^  (0.0098 


(0.0650)' 


I  2(0.0714) 
0.06^ 


S  + 


_(4.27) 

Airplane  Transfer  Function 


_  +  2(0-^95)  _  +  . 
2  4.27  ®  ^ 


m 


^ife  +  ■') 


/-L 


2(0-7) 


V50/  50 


s  +  1 


(1) 


Controller  Transfer 
Function 


The  Bode  diagram  for  G(ju))  is  shown  in  Fig.  2.  The  closed- loop  system  has  three 
regions  of  interest  defined  by 


(a)  |G(jCD)|»1,  over  which 


G(M. 


1  +  G(ja)) 
G(jaj) 


=  1 


(b)  |G(ja))|  «lj  over  which  [  |  =  |G(ja)) 

(c)  |G(jm)|  1  1 


The  form  of  the  closed-loop  transfer  function,  in  this  last 

region  defines  the  dominant  modes  of  the  closed-loop- system  dynamic  response 
for  impulse  and  step  inputs.  In  most  cases  G(jcD)/[|l  +  G(j(D)J  in  the  region  where 
|G(ja))|  is  of  the  order  of  unity  can  be  approximated  by  a  first-,  second-,  or 
bb.ird— order  system,  the  modes  of  which  will  deteimine  the  major  features  of  the 
response.  The  open- loop  amplitude  asymptotes  of  an  appropriate  equivalent  system 
for  this  example  are  shown  in  Fig.  2. 
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Figure  2.  Open- Loop  Bode  Diagram  of  G(ja))  Transfer  Function 
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Applying  this  approximation  to  the  present  exanple  yields  the  closed- loop 
(jco)  Bode  diagram  of  Fig.  3.  The  Bode  diagram  for  the  exact  closed-loop  system 
is  also  shown  for  comparative  puip)Oses.  It  will  he  observed  that  the  error  of 
the  approximation  is  small.  If  greater  accuracy  is  required,  more  complicated 
open- loop  equivalent  systems  can  be  produced  by  retaining  more  of  the  terms  in 
the  complete  open- loop  transfer  function. 


In  this  example  the  crossover  frequency,  is  of  the  order  of  the  servo 
break  frequency  (50  rad/sec).  Usually  this  frequency  will  be  »cu^;  the  effect 
of  the  associated  high-frequency  leads  and  lags  can  then  be  approximated  by 
replacing  them  in  either  the  open-  or  closed-loop  transfer  factions  by  a  pure 
time  delay  term,  A  satisfactory  approximation  for  the  time  delay  is 

=  -(^leads  -  '^laffs)  1 )  • 

leans  lags  frequency 

In  general,  airplane  transfer  function  break  frequencies  and  time  constants 
are  spaced  so  that  G(j(u)  in  the  region  of  crossover  can  be  satisfactorily  approx¬ 
imated  by  a  system  of  not  more  than  fourth  order. 


The  artifice  of  replacing  the  actual  flight  control  system  by  the  equivalent 
system  is  extremely  valuable  in  simplifying  analysis  and  optimization  procedures. 
Such  simplification  is  usually  necessary  to  prevent  the  physical  realities  being 
submerged  by  a  flood  of  mathematics. 


C.  ASSESSMENT  OF  PERFORMANCE  CRITERIA 


The  basic  requirements  that  a  criterion  must  satisfy  if  it  is  to  be  of 
practical  use  are  defined  in  Ref.  ^  as 

1  .  validity 

2.  selectivity 

3.  ease  of  application 

These  will  now  be  reviewed,  and  the  concept  of  "compatibility"  introduced.  As 
will  be  shown,  this  concept  is  useful  in  determining  the  validity  of  random  input 
performance  criteria  for  flight  control  systems. 

Validity 

Validity  means  that  the  criterion  yields  systems  that  have  desirable  perform¬ 
ance  characteristics  for  the  input  environment  of  interest.  Desirable  character¬ 
istics  would  include  adequate  phase  margin,  realizable  bandwidth,  and,  for  a 
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Figure  J-  Comparison  of  Exact  and  Approximate  Closed-Loop  Bode  Diagrams 
of  G(Jcjd)/'i  +  C(ja:i)  Transfer  Function 
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Phase  Angle,  deg 


transient  input  environment ,  good  step  response.  Although  no  simple  form  of 
step  response  can  be  explicitly  defined  as  a  iiniversally  applicable  optimum^  it 
is  generally  accepted  that  fast  rise  time^  small  overshoot,  and  lov  settling 
time  are  desirable.  (For  full  discussion  of  these  aspects  of  system  performance 
see  Ref.  1  .)  A  second-order  unit  numerator  system  having  a  damping  ratio,  of 
about  0.7  satisfies  these  requirements  quite  well. 

At  first  sight  it  might  seem  that  an  assessment  of  system  merit  based  on 
step  (or  other  transient)  response  has  little  relevance  to  the  value  of  that 
system  for  random  inputs.  However,  flight  control  systems  must  be  satisfactory 
for  a  wide  range  of  inputs,  some  (such  as  gusts)  essentially  of  a  random  charac¬ 
ter,  others  (such  as  engine  failures,  pilot  commands)  steplike,  and  primarily 
deterministic.  A  random  input  performance  criterion  that  yields  a  system  having 
a  very  lightly  damped  step  response  cannot,  therefore,  be  regarded  as  valid  for 
flight  control  systems.  Ideally,  a  random  input  performance  criterion  applied 
to  a  second-order  unit  numerator  system  should  yield  a  damping  ratio  of  approxi¬ 
mately  0.7*  This  ideal  provides  a  yardstick  by  which  most  of  the  criteria 
discussed  in  this  report  will  be  assessed.  Although  good  step  response  is  a 
necessary,  rather  than  sufficient,  condition  for  validity,  criteria  that  fall 
far  short  of  this  standard  can  be  rejected  without  further  study,  particularly 
since  many  equivalent  flight  control  systems  are  only  of  second  order. 

To  formalize  the  assessment  of  system  merit  on  the  basis  of  response  to  both 
deterministic  and  random  inputs,  the  concept  of  "compatibility"  will  now  be 
introduced.  Consider  a  deterministic  input,  rp(t),  and  a  stationaiy  random  input, 
rj^(t) .  Let  these  inputs  be  applied  to  identical  linear  constant-coefficient 
systems  having  the  transfer  function  H(s),  and  let  Mp  and  Mp  be  appropriate 
random  and  deterministic  input  performance  measures,  respectively.  If,  for  any 
specified  rj^(t) ,  an  rj^(t)  can  be  found  such  that  1%  =  Mp,  for  all  H(s),  then  Mp 
and  are  said  to  be  compatible  and  rj^(t)  is  called  the  transient  analog  of 
rp(t) .  (For  compatibility  only  the  numerical  values  of  and  Mp  need  be  equal; 
their  dimensions  (units)  may  differ.) 

A  brief  exsanple  of  a  compatible  criteria  is  given  on  p.  15*  A  more  detailed 
discussion  of  compatible  criteria  and  transient  analogs  for  both  signal  and  noise 
is  given  in  Chapter  IV.  The  use  of  transient  analog  inputs  and  coii5)atible  per¬ 
formance  measures  enables  analysis  of  systems  with  random  input  quantities  to  be 
replaced  by  analysis  with  detemninistic  input  quantities  (such  as  steps  and  ramps). 
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which  are  much  more  easily  visualized.  Compatibility  is  thus  a  desirable 
qmntity.  It  does  not  justify  criteria;  the  prime  requirement  is  still  validity. 
However^  the  compatibility  concept  could  be  used  to  guide  the  search  for  a  valid 
random  input  performance  criterion.  Instead  of  trying  various  random  criteria 
and  then  checking  their  (possibly  imsatisfactory)  validity^  one  could  adopt  the 
following  direct  procedure: 

1  .  find  a  valid  deterministic  criterion 
2.  find  its  compatible  random  form 

Neither  of  these  tasks  may  be  easy.  Nevertheless^  it  is  hoped  that  this  system¬ 
atic  approach  will  be  more  successful  than  the  usual  procedure  'of  suggesting 
random  input  criteria  without  consideration  of  the  deterministic  response  char¬ 
acteristics  of  the  resulting  system. 

Some  physical  significance  can  be  attached  to  compatibility.  It  can  be 
argued  that  the  separation  of  flight  control  system  inputs  into  "random"  and 
"deterministic"  categories  is  too  arbitrary.  For  exan^le^  pilot  inputs  can 
often  be  well  approximated  by  step  movements  of  the  controls^  the  amplitude  and 
timing  of  these  steps  being  random.  Suppose  that  in  such  an  input  the  delay 
between  each  successive  step  greatly  exceeds  the  system  settling  time  so  that 
the  system  error  essentially  settles  to  zero  between  successive  steps.  This 
input  could  be  thought  of 

1 .  as  a  noise -free  stationary  random  input 

2.  as  a  stationary  sequence  of  steps  commencing 
at  time  tQ  and  continuing  to  t  =  oo. 

One  method  of  optimizing  the  system  would  be  to  apply  some  standard  deterministic 
input  performance  criterion  to  each  member  of  the  sequence  of  step  responses.  If 
this  deterministic  criterion  is  valid  (as  it  should  be) ^  the  optimized  system 
step  response  would  resemble  the  step  response  of  a  second-order  unit  numerator 
system  with  a  damping  ratio  of  about  0.7* 

Finding  a  valid  compatible  random  criterion  is  much  more  difficult  than  it 
may  appear  at  first  glance.  Many  deterministic  performance  criteria  do  not  have 
compatible  forms ^  and  some  that  are  compatible  fall  short  of  the  ideal  as  regards 
validity.  This  last  point  will  now  be  demonstrated  by  considering  the  and  lE^ 
performance  measures. 
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for  stationary  random  inputs 


e2 


lim  _1^ 
T-^oo  2T 


IE 


2 


for  deterministic  inputs  commencing  at  t  =  0 


It  will  now  be  shown  that  while  these  measures  are  compatible^  their  validity 
is  less  than  ideal.  In  order  to  emphasize  the  physical  significance  of  compati¬ 
bility^  only  the  special  class  of  random  inputs  described  on  p.  12  will  be  con¬ 
sidered.  (A  general  proof  of  the  compatibility  of  and  lE^  for  both  correlated 
and  uncorrelated  signal  and  noise  inputs  is  given  in  Chapter  IV.) 

Example  of  Compatible  Criteria.  It  is  clear  that  the  minimum  lE^  criterion 
cannot  be  directly  applied  to  stationary  random  quantities  because  fluctuates 
about  a  constant  level  and  the  integral  would  fail  to  converge. 

Consider  the  special  input  described  on  p.  12^  consisting  of  steps  of  random 
amplitude  occurring  at  random  intervals  substantially  exceeding  the  system 
settling  time.  Evaluating  lE^  for  each  of  the  step  responses  yields 


+ 


2 


dt 


where  tQ  is  the  time  of  application  of  the 
first  step 

t-j  is  the  time  of  application  of  the 
second  step 

t  -1  is  the  time  of  application  of  the 
nth  step 


tn  =  tj^-i  +  an  arbitrary  finite  time 
much  greater  than  the  system 
settling  time 


If  the  input  is  stationary^  this  sum  becomes  the  simi  of  an  infinite  number  of 
integrals^  and  hence  fails  to  be  of  use  in  practical  calculations.  In  order  to 
obtain  a  finite  measure,  the  time  average  value  of  these  integrals  will  be  taken, 
and  the  performance  measure  modified  to 
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^  -  V,)  ^  -  (t,  -  to)  ^ 


€^dt 


Coalescing  the  limits  of  integration  (see  p.  101  of  Ref.  6  and  p.  1  56  of 
Ref.  8  for  formal  justification  of  this  step)^  the  criterion  becomes 


minimum 


_J _  f 

^n  ■  "^0  J+ 


n 


G^dt 


0 


For  a  stationary  situation,  tQ  -00  and  tji  +00,  and  the  criterion  can 
be  expressed  in  the  familiar  form 


minimum 


lim 

T^oo  2T  J_^ 


e^dt 


minlmtun 


(2) 


Having  related  and  lE^  by  elementary  physical  considerations,  one  would  expect 
that  both  the  minimum  and  the  minimum  lE^  criteria  would  be  compatible.  This 
can  be  quickly  proved  for  a  unit  numerator  second-order  system  (although  the  proof 
can  be  generalized  for  any  system) .  The  required  system  transfer  functions  are, 
for  unity  feedback. 


H(s)  = 


C(s) 


E(s) 


s^  +  2^s  +  1  ^ 


+  2t;s 


+2^5+1 


As  has  been  shown  in  Ref.  1  ,  for  a  unit  step  input. 


(5) 


lE^ 


lE^  is  minimized  by  ^  =  0.5-  As  shown  on  p.  l4l  of  Ref.  8,  the  special  random 
input  of  p.  12  could  be  described  in  conventional  statistical  teims  by  a  power 
spectrum  of  the  form 


lim  1 
^  ^  a^  + 


'ik 


(5) 


Evaluating  with  the  above  input  applied  to  the  H(s)  of  Eq  3  yields 


€2 


lim  1 
a  0  j 


<J>gg(s)ds 


(6) 


lim  1 

^  r  (s2  +  2cs)  1 

r  (s2  -  2^s)  I 

a  —0  2rtj 

x>  +  l)(a  +  s)J 

[(s^  -  2^s  +  l)(a  -  s)J 

ds 


This  can  Jbe  evaluated  using  the  tabulated  integral  forms  of  Appendix  Ref.  6^ 
whence 

■^2  =  ^  +  jL  ^  Ie2  (^) 

Thus  the  and  lE^  criteria  are  compatible.  A  general  proof  (applicable  to  all 
stationary  random  inputs)  showing  that  these  performance  measures  are  compatible 
for  any  linear  system  is  given  in  Chapter  IV.  The  value  of  this  coii5)atibility 
is  rather  diminished  by  the  fact  that  minimum  lE^  is  generally  regaI^ied  as  only 
moderately  valid  because  the  ^  =  0.5  step  response  has  a  relatively  high  over¬ 
shoot  and  a  long  settling  time  compared  to^  say^  the  optimum  ITAE  second-crder 
unit  numerator  system  =  O.76)  . 

It  is  interesting  to  examine  the  difficulties  in  constructing  a  compatible 
criterion  for  ITAE.  With  the  same  input  as  above ^  applying  the  ITAE  criterion 
to  each  of  the  step  responses  yields  the  criterion 


mininiuTn 


e  dt  +  minimum 


dt  +  •  •  •  +  minimum 


dt 


Taking  the  time  average  value  of  these  integrals,  the  criterion  may  be  modified 
to 


minimum 


1 1  €  I dt  + 


There  appears  to  be  no  standard  random  form  to  which  this  expression  can  be  con¬ 
densed,  i.e.,  no  random  criterion  compatible  with  ITAE  has  been  foimd.  Of  course. 
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this  is  not  a  proof  that  no  compatible  criterion  exists;  if  one  could  be  founds 
it  would  be  extremely  valuable.  Further  discussion  of  compatible  criteria  is 
given  in  Chapter  IV. 

Other  Requirements  for  Ferfoi:mance  Criteria 

Validity  has  been  discussed  at  some  length;  the  remaining  requirements  of 
"selectivity”  and  "ease  of  application"  can  be  dealt  with  more  briefly. 

"Selectivity"  implies  shaipi  differentiation  between  "good"  systems  and  those 
which  are  merely  'acceptable."  Of  the  performance  measures  examined  in  Ref. 

ITAE  is  particularly  satisfactory  in  this  respect.  For  a  step  input  to  a  second- 
order  unit  numerator  system^  the  minimum  ITAE  is  1  .96  at  ^  =0.76^  rising  to 
2.25  at  ^  =  0.61  and  ^  =  O.9O.  Since  the  ITAE  criterion  maintains  good  selec¬ 
tivity  for  high-order  systems  and  has  been  favorably  received  in  almost  all 
relevant  references  (e.g..  Ref.  31  and  35),  it  will  be  taken  as  setting  an 
acceptable  standard  for  selectivity. 

Ease  of  application  demands  that  the  criterion  should  be  readily  expres¬ 
sible  in  terns  of  system  parameters,  and  that  convenient  procedures  for  its 
evaluation  should  exist. 
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CHAPTER  I 


CRITERIA  DEFINED  SOLELY  AS  FUNCTIONS  OF  ERROR 

This  chapter  is  concerned  with  performance  measures  expressible  in  the  form 
fCe),  the  sole  independent  variable  being  e,  the  error  between  the  actual  system 
output  and  a  specified  desired  output.  The  majority  of  the  criteria  that  have 
been  proposed  for  linear  constant -coefficient  systems  with  random  inputs  are  of 
this  form.  In  particular^  the  criterion  of  mini  Timm  mean  square  error  has  been 
extensively  studied  for  stationary  random  inputs .  Outstanding  contributions  to 
the  study  of  this  criterion  have  been  made  by  Wiener  (Ref.  4)  and  Phillips 
(Ref.  3),  and  throxjghout  this  report  reference  will  frequently  be  made  to  Wiener 
optimization  and  Phillips-type  optimization.  Therefore^  for  ease  of  reference^ 
the  Wiener  and  Phillips-type  optimization  procedures  are  briefly  summarized  below. 
The  essential  difference  between  the  procedures  is  that  in  Phillips-type  optimiza¬ 
tion  the  form  of  the  system  (the  order  of  the  transfer  function  numerator  and 
denominator)  is  prescribed  by  the  analyst ^  whereas  the  form  of  the  Wiener  system 
depends  only  upon  the  spectra  describing  the  input  environment  and  desired  output. 


A.  BRIEF  SUMMARY  OF  WIENER  OPTIMIZATION 


Wiener  investigated  the  problem  of  recovering  a  stationary  random  signal,  f(t), 
from  an  input  comprised  of  signal  plus  noise,  n(t) ,  when  the  desired  output,  C(j(t), 
is  equal  to  the  signal  advanced  or  retarded  by  t]  sec.  The  cases  where  t)  >  0, 

T)  —  0,  and  T]  0  are  referred  to  as  prediction,  smoothing,  and  inteipolation 
(or  lagging) ,  respectively.  No  restrictions  are  imposed  other  than  that  the 
optimum  system  should  be  linear  and  physically  realizable,  i.e.,  no  output  can 
arise  without  a  prior  input.  A  block  diagram  representation  of  this  problem  is 
given  in  Fig.  4.  Components  are  indicated  by  their  transfer  functions,  for  brevity. 


€(t) 


Figure  4 .  Block  Diagram  for  Wiener  Optimization 
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H(s)  is  the  transfer  function  of  the  system  under  consideration^  and  its  output, 
c(t),  is  denoted  hy  c^(t)  when  H(s)  becomes  %(  s),  the  Wiener  (optimum  linear) 
system.  Using  the  ergodic  hypothesis,  the  time  average  square  error  equals  the 
ensemble  average  square  error 

=  E(ca  -  cf  (8) 

where  G(t)  =  c^(t)  -  c(t),  the  difference  between  desired 
and  actual  outputs,  and  E  denotes  the  ensemble  average 


Let  Hy(s)  =  >C[hy(t)3  denote  the  physically  realizable  linear  constant  coefficient 

system  which  minimizes  %(s)  is  found  by  considering  any  other  realizable 

linear  system,  h^(t)  +  ag(t);  and  imposing  the  condition  that  for  all  a  (a  param¬ 
eter)  and  any  g(t),  this  system  yields  a  larger  than  that  due  to  i.e.. 


-  Cw  -  acg)2  -  e(c^  -  Cy)2  >  0  (9) 

where  Cg(t)  is  the  additional  output  due  to  g(t),  and 
c^(t)  is  the  output  of  the  Wiener  system 


Expanding  Eq  9  gives 


E(C(3^  -  -  2aE|^(C(i  -  Cy)cgJ  +  a%(c|)  -  E(c^  -  >  0 


(10) 


-2aE^(cd^  -  c^)cgj  +  a%(c 


;)  >  0 


(11) 


Because  the  second  term  is  always  positive,  the  inequality  can  hold  for  all 
a  only  if 


E 


=  0 


(12) 
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This  is  the  necessary  and  sufficient  condition  for  Cy  to  be  the  output  of  the 
linear  system  that  yields  minimum  e^.  Using  the  convolution  relationship  for 
physically  realizable  linear  systems ^ 

fee 

Cg(t)  =  I  g(T)r(t  -  T)dT  (15) 

Jo 


Equation  12  can  be  written 


Interchanging  integrations^ 


a 


0 


(1^) 


(15) 


where  the  crosscorrelation  is  defined  as 

/+T 

a(t)b(t  +  T)dt  =  E[a(t)b(t  +  t)] 

T 

where  a  and  b  are  general  stationary  random  signals 

Equation  15  will  be  true  for  all  g(T),  r  >  0,  and  all  a,  only  if  the  expression 
in  braces  is  equal  to  zero,  i.e.^ 


^rC(3^('’')  ~  ^rcy(’^) 


T  >  0 


(16) 


Usually  the  desired  output  is  either  the  signal  or  a  prediction  of  this  signal 
T]  sec  in  the  future : 
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Cd(t)  =  f(t  +  T]) 


(17)  • 


Because  Cy(t)  = 
gral  equation: 


u)r(t  -  u)du^ 


Eql6  becomes  the  familiar  Wiener-Hopf  inte- 


/•oo 

+  n)  =  I  hw(u)Rj.^(T  -  u)du  t  >  0  (l8) 

Jo 


The  solution  of  Eq  1  8  yields  the  optimum  linear  system. 
Eq  l8  can  be  expressed  in  the  frequency  domain  as  (Ref.  10, 


Alternatively, 
P-  92) 


1 

(19) 


where 


<i> 

<5 


[] 


+ 


the  cross-spectrum  between  the  desired 
output  and  the  sum  of  signal  and  noise 

a  factor  of  containing  all  the 

left-half- s-plane  poles  and  zeros  of  <E>^p(a)) 

^rr^'^^^*^rr^'^^^  input  power  spectmim) 

means  expand  in  partial  fractions,  and 
then  keep  only  terms  with  left-half -plane 
poles 


Equations  1  8  and  1  9  constitute  the  principal  results  of  the  Wiener  optimization 
theory. 

B.  BRIEF  SUMVIARY  OF  PHILLIPS-TYPE  OPTIMIZATION 


The  method  of  optimization  developed  by  Phillips  differs  from  that  described 
above  in  that  the  form  of  Hp(s),  the  Phillips  optimum  system  transfer  function, 
is  prescribed.  The  error  power  spectrum  is  calculated  for  general  values  of  the 
numerator  and  denominator  coefficients  of  Hp(s).  The  mean  square  error  is  then 
obtainable  from  the  following  relationship: 
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(20) 


—  1 

-  J_  I  <j)  (a3)(ia) 

^  2rt  I 

^-00 

The  evaliiation  of  this  integral  is  facilitated  by  the  use  of  integral  tables 
given  by  Phillips  (Ref.  3  and  in  a  more  convenient  form  in  Ref.  6,  and  56). 

The  mean  square  error  is  thus  obtained  in  literal  terms  involving  the  coefficients 

of  Hp(s),  and  the  resulting  expression  is  then  minimized  with  respect  to  these 

coefficients . 

To  illustrate  a  typical  Phillips-type  optimization,  a  second-order  zero- 
position-error  system  will  be  considered;  its  transfer  function  is 


H(s) 


1 

s^  +  2^s  +  1 


The  signal  power  spectrum  is  ^fAs)  =  —5 - 5,  the  noise  is  zero,  and  the  desired 

-s^  +  a^ 

output  is  equal  to  the  signal.  The  resulting  mean  square  error  has  been  graphed 
in  Fig.  5.  For  a  specified  value  of  a,  the  Phillips  filter  has  the  ^  indicated 
by  the  dotted  line.  In  this  example,  the  mean  square  error  criterion  is  unselec- 
tive  in  that  the  off -optimum  mean  square  error  is  very  little  greater  than  the 
minimum.  However,  discussion  of  the  merit  of  as  a  performance  criterion 
relative  to  other  criteria  will  be  delayed  until  Chapter  III,  after  demonstrating 
that  many  apparent  alternative  criteria  are  in  fact  equivalent  to  minimum  mean 
square  error. 

C.  DETERMIHATIOH  OF  THE  MINIMUM  MEAN  SQUARE  ERROR  SYSTEM 

WITHOUT  RESTRICTION  OF  LINEARITY  OR  CONSTANCY  OF  COEFFICIENTS 

When  the  signal  and  noise  are  stationary  random  processes,  the  Wiener  system, 
by  definition,  yields  a  lower  mean  square  error  than  any  other  physically  realiz¬ 
able  linear  constant -coefficient  system.  It  is  logical  to  inquire  whether  a 
time-varying  and/or  nonlinear  physically  realizable  system  could  yield  a  smaller 
mean  sqixare  error  in  such  an  input  environment.  Strictly,  the  answer  to  this 
question  lies  beyond  the  scope  of  this  report,  which  is  primarily  concerned  with 
time- invariant  linear  systems;  nonlinear  systems  are  usually  studied  separately 
because  their  analysis  demands  specialized  techniques,  and  because  the  generality 
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Figure  5.  Effect  of  Input  Break  Frequency  on  Mean  Square  Error  of  a  Second-Order  Unit  Numerator  System 


of  the  results  obtained  is  limited.  (For  example,  the  response  of  a  nonlinear 
system  to  a  deterministic  input  may  change  radically  with  a  small  change  in  the 
input  amplitude.)  However,  for  stationary  random  inputs,  it  is  sometimes 
possible  to  obtain  quite  useful  and  general  results  without  imposing  the  restric¬ 
tion  of  linearity.  In  particular,  it  can  be  shown  that  with  a  stationary  random 
Gaussian  input  environment,  the  Wiener  system  yields  a  lower  mean  sq-uare  error 
than  any  other  physically  realizable  system,  linear  or  nonlinear,  with  or  without 
time-varying  coefficients. 

In  order  to  demonstrate  this  result,  a  brief  departure  from  purely  linear 
analysis  is  necessary.  This  is  amply  justified  by  the  importance  of  the  result 
obtained. 

Formulation  of  the  Ensemble  Mean  Square  Error 


It  is  necessary  first  to  consider  a  general  (i.e.,  not  necessarily  Gaussian) 
input  environment,  and  to  derive  the  formula  for  the  minimum  mean  square  error 
system.  Sherman  (Ref.  1  3)  quoted  this  formula,  but  did  not  prove  it.  In  fact, 
no  proof  was  found  in  any  of  the  references  consulted  by  the  authors  of  this 
report.  The  derivation  given  below  was  obtained  by  utilizing  some  results  given 
by  Cramer  (Ref.  57)* 

Consider  the  system  j.llustrated  in  Fig.  k  (p.  17)  where  now  the  sole  restric¬ 
tion  upon  the  output,  c,  is  that  it  be  the  output  of  a  physically  realizable 
system;  i.e.,  c  depends  only  upon  the  past  and  present  values  of  r.  The  system 
is  permitted  to  be  linear  or  nonlinear  and  time-varying  or  time- invariant .  The 
ensemble  mean  square  error  is 

/CO  rco 

I  (Cd  -  c)S(ca.^  c)dCd(ic  (21) 

■ooj-oo 

where  p(c^,  c)  is  the  joint  proba¬ 
bility  density  function  of  c^^^  and  c 

So  that  Cramer *s  results  can  be  used  without  modification,  it  will  be  assxamed 
that  the  system  operates  only  upon  sampled  values  of  r(t)  available  only  at  a 
finite  number,  n,  of  sampling  instants  prior  to  t,  the  present  time.  For  this 
condition,  c(t)  will  be  some  function,  of  the  present  and  past  input  samples; 
i.e. , 
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c(t) 


(22) 


t(rQ, 

i-T,  rg, 

^n) 

where 

^0  = 

r(t) 

ri  = 

r(t 

-  T) 

r2  = 

r(t 

-  2T) 

^n  = 

r(t 

-  nT) 

T  is  the  sampling  period 


Making  use  of  Eq  22  the  ensemble  mean  square  error^  Eq  21  ^  can  be  vritten  as 


2  fOO  2 

E(c^  -  c)  =  I  ...  I  -  c)  ro,  r-,  •••  rn)dcd.dro<iri  dr^  (23) 

y -OoJ-00 


where  pCc^^^^  rQ^  ^  rj^)  is  the  joint  probability 

density  function  of  c^^  and  all  the  past  sampled  values 
of  r  that  contribute  to  c 


The  joint  probability  density  function  ^l  ^  rewritten  as 


p(cd^  r)  =  p(c4/r)p(r) 


(24) 


where  'r'  denotes  the  sequence  of  past  sampled  input 
values,  Tq,  r-,  ,  rg, 

p(cd/r')  is  the  conditional  probability  density 
function  defined 

the  probability  density  function  of  c^^  assum¬ 
ing  that  the  particular  input  sequence  has 
occurred. 


Inserting  Eq  24  into  Eq  25^  and  rearranging  terms ^  yields 


E(cd  -  c)‘ 


/GO  rco 

I  ^ 

■GO  J  -OO 


/OO 

' 

OO 


drodr^  •••  drnp(^^)  I  (c^  -  c)  p(C(i/r)dC(3^  (25) 
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Minimization  of  the  Ensemble  Mean  Sqxxare  Error 


2 

The  minimization  of  E(ca.  -  c)  follows  the  procedure  in  Ref.  37^  p.  271-272. 


The  fo.lJ owing  points  about  Eq  25  should  be  noted: 

1  .  c  occurs  only  in  the  inner  integral  over  c^^^ 

2.  because  c  is  a  function  of  r*  alone ^  it  is  constant  when 
integrating  over 

3-  the  integral  over  is  the  second  moment  of  c^  taken 
about  c^  given  that  a  general  r*  has  occurred. 

Equation  25  will  be  minimized  if  c  is  chosen  such  that  the  integral  over  c^ 
is  minimized  at  each  value  of  "f"  that  can  possibly  arise  from  the  given  signal 
and  noise.  The  well-known  fact  that  the  second  moment  is  minimized  when  taken 
about  the  mean  implies  that  c  must  be  equal  to  the  conditional  mean  of  c^^ 
given  r^  i.e.^ 


(26) 


C 


This  is  the  general  formula  for  the  absolute  optimum  mean  square  error  system. 

Inteipretation  of  Eg  26  for  General  Inputs.  The  meaning  of  Eq  26  can  best 
be  understood  by  considering  the  case  where  c^^^  equals  the  signal;  as  usual^ 
r  is  the  sum  of  signal  and  noise.  The  present  and  past  sampled  values  of  r^ 
relative  to  the  present  time,  t,  are  available  as  inputs  to  the  absolute  optimum 
system.  From  these  observations  on  the  sum  of  signal  and  noise,  the  best  esti¬ 
mate  of  the  signal  alone  is  the  ensemble  average  value  of  the  signal  at  time,  t, 
utilizing  the  actual  sequence,  *?,  that  has  occurred.  Tbe  formula  for  the  absolute 
optimum  system  (Eq  26)  is  intuitively  satisfying  because  it  makes  use  of  all  the 
information  that  observations  on  the  sum  of  signal  and  noise  have  conveyed  about 
the  signal  alone;  i.e.,  p(C(3^/r).  Unfortunately,  the  determination  of  the  condi¬ 
tional  mean  is  in  general  very  difficult  if  the  signals  involved  are  non-Gaussian. 

Interpretation  of  Eq  26  for  Gaussian  Signal  and  Uoise.  When  the  signal  and 
noise  are  Gaussian,  both  c^^  and  r  are  Gaussian.  In  particular,  the  sequence 
^  “  ^0^  ^1  ’  ’  *  ^n  (defined  in  Eq  24)  has  a  multivariate  Gaussian  joint  probability 
function.  Cramer  (Ref.  37,  p.  31^^  5^5)  has  shown  that  conditional  density 
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functions  of  Gaussian  signals  are  Gaussian.  In  addition^  he  has  shown  that  the 
conditional  mean  is  a  linear  function  of  the  given  variables  r'  =  Tq^  r-j  ^  •••  r^. 

It  is  now  permissible  to  allow  n^  the  number  of  sampled  values^  to  become  arbi¬ 
trarily  large  and  the  sampling  period  to  become  as  small  as  may  be  desired. 

Neither  of  these  limiting  processes  will  affect  the  linearity  of  the  absolute 
optimiom  system.  Therefore,  the  absolute  optimum  system  is  linear,  and  if  the 
Gaussian  signal  and  noise  are  also  stationary  then,  by  definition,  this  system 
must  be  the  Wiener  system. 

D.  SUMMARY  OF  RESULTS  ON  MEAN  SQUARE  ERROR  CRITERIA 

The  results  obtained  on  the  Wiener,  Phillips,  and  absolute  minimum  mean  square 
error  systems  are  summarized  on  the  "family  tree"  of  Fig.  6,  which  shows  how  these 
systems  are  related.  It  will  be  demonstrated  later  in  this  report  that  many  other 
criteria  yield  related  systems,  and  that  many  of  these  latter  systems  are  simply 
minimum  mean  square  error  systems.  In  particular,  family  trees  for  criteria 
expressible  as  f(€)  and  for  time-weighted  criteria  are  given  on  pp.  28  and  63, 
respectively.  (it  would  be  possible  to  join  appropriate  branches  from  each  of 
these  trees.  However,  the  resulting  single  tree  would  have  many  branches,  result¬ 
ing  in  a  fairly  complicated  presentation  which  would  tend  to  obscure  the  simplicity 
of  many  of  the  relationships  involved.) 

E.  BENEDICT  AND  SONDHI*S  PERFORMANCE  MEASURE  (Applicable  to  Stationary 

Gaussian  Processes  Only,  Both  Wiener  and  Phillips-Type  Optimization) 

For  stationary  Gaussian  processes,  Benedict  and  Sondhi  (Ref.  38)  have  shown 
that  the  linear  system  (Wiener  or  Phillips-type)  which  minimizes  also  minimizes 
any  performance  measure  of  the  form 


f(i€i)  =  El^r^  =  (27) 

k  k 


where  is  positive  (but 
not  necessarily  an  integer) 
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Figure  6.  Suinmary  of  Minimum  Mean  Square  Error  Systems 
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Figure  7*  Summary  of  Minimum  f ( e)  Systems 


Proof 


Consider  a  general  example 


f(|e|)  =  +  |€|^2  +  |e|“5  +  ...  (28) 

where  ^  112^  n^  •  •  •  are  real  and  positive  integers  or 
nonintegers  (rational  or  nonrat ional)  and  the  number 
of  used  is  arbitrary 

Each  term  |€|^^  can  be  evaluated  using  the  ensemble  average^  which  for  Gaussian 
processes  is 


(29) 


Using  the  integral  tables  of  Ref.  59  ^  P*  201^ 


pk  _ 


(30) 


where  o  =  yr  as  usual 

r(x)  =  the  Gamma  function 


Substituting  Eq  JO  into  Eq  28  yields 


_  P%/2 

f(UI)  =  E  ^  r 

k 

From  Eq  Jl  i't  is  apparent  that  f(|G|)  can  be  minimized  only  by  minimizing  a- 
Therefore,  the  system  that  minimizes  minimizes  all  possible  forms  of  f(|G|) 
as  defined  in  Eq  27  and  28. 
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Generalization  of  Benedict  and  Sondhi’s  Result 


It  can  be  shown  that  f(|€|)  is  a  nondecreasing  function  of  €.  The  slope  is 


d€ 


E 


(32) 


which  for  e  >  0  is  always  positive  and  nonzero-  It  would  appear  that  one  could 
construct  a  great  variety  of  f(lG|)*s  by  appropriate  choice  of  the  k*s  and  nj^*s. 
A  typical  f(|€|)  is  sketched  below. 


Figure  8.  Typical  Nondecreasing  f(e)  Loss  Function 


Note  that  f(|€|)  could  be  generalized  to  the  form 


f(lel)  =  S 

k  ^ 


(33) 


without  changing  the  fact  that  f(|e|)  is  minimized  by  minimizing  e^. 
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All  the  above  results  could  have  been  derived  using  the  analyses  of  Sherman 
and  Chang  presented  later  in  this  chapter.  The  original  derivation  of  Benedict 
and  Sondhi  was  given  because  it  is  particularly  easy  to  follow  and  forms  a  useful 
preparation  for  the  discussion  of  Sheman^s  results  that  now  follows. 

F.  SHERMAN  *S  PERFORMANCE  MEASURE 

Benedict  and  Sondhi  (Ref.  38)  considered  a  fairly  general  f(6)  with  station¬ 
ary  Gaussian  random  signals  and  found  that  f (c)  was  minimized  by  minimizing  e^. 
The  question  naturally  arises  as  to  whether  or  not  a  similar  simplification 
occurs  when  the  signal  and  noise  are  nonstationary*  and/or  non-Gaussian  random 
processes.  This  problem  has  been  solved  by  Sherman  (Ref.  15)^  who  showed  that 
minimizing  E  defined  below^  is  accomplished  by  minimizing  E^e^J  .  (Ensemble 

averages  are  used  since  the  ergodic  hypothesis  will  not  hold  if  the  input  environ- 
is  nonstationary.)  Many  other  performance  measures  can  be  expressed  as  special 
cases  of  Sherman *s  measure.  These  special  cases  are  detailed  below^  and  their 
interrelationships  are  summarized  on  the  "family  tree"  on  p.  28. 

Sherman's  "loss  function^"  f(e)^  has  the  following  properties: 

f(0)  =  0 

f(e)  =  f(-€) 

f(€2)  >  f(ei)  >  0 
for  €2  ^  G]  ^0 

This  1(0)  is  similar  to  the  loss  function  of  Benedict  and  Sondhi  (Ref.  38) 
(illustrated  in  Fig.  3  )  in  that  f(G)  must  never  decrease  for  increasing  [el,  but 
it  differs  in  that  it  may  have  zero  slopes  at  some  points  or  in  some  regions. 

The  optimum  system  should  minimize  the  ensemble  average  loss^  which  is 


null  property 
symmetry  property 


(54) 


monotonic  property 


E[f(e)] 


f(cd  -  c)p(C(3^,  c)dc^dc 


(35) 


By  a  line  of  reasoning  similar  to  that  in  Section  C,  Eq  35  can  be  written  in  a 
form  similar  to  Eq  26.  In  the  derivation  of  Eq  36,  one  merely  substitutes  f(e) 


51 


for  and  then  the  equation  corresponding  to  Eq  25  for  Ejf(€ 

rco  rco  rco 

E[f(c^  -  c)]  =  /  ‘I  p(r)drodrT  dr^  I  f(cd.  -  c)p(c^/^-)dC(i  (36) 

Joo  •/-OO  J-00 

The  system  that  minimizes  Eq  36  depends  on  the  particular  conditional  density 
function^  p(C(^/r)^  that  results  from  the  given  signal  and  noise  distribution. 

Nowy  if  the  conditional  probability  density  function^  pC^d/r)^  satisfies  the 
following  sufficient  conditions  (derived  from  some  more  general  conditions  stated 
by  Sherman^  Ref-  15)^ 

1  -  symmetric  about  the  origin  of  c^^^ 

2.  has  its  only  maximum  there 
3-  is  continuous 

then  a  lemma  in  probability  theory  quoted  by  Sherman  states  that  E  {f(cd  -  c)/rj 
is  minimized  when  c  equals  the  conditional  mean  of  c^^^  given  r^^  i.e.^ 

c  =  E[cd/r]  (37) 


where  c  = 

c(t) 

^d  = 

Cd(t) 

r  = 

the  sequence  rQ,  r-j  ,  r2  •  •  •  r^^ 
as  defined  in  Section  C,  Eq  24 

Equation  37  describes  the  system  that  yields  the  absolute  minimum  E[f(e)2  at  the 
present  time^  t.  This  is  the  same  system  that  minimizes  the  which^  as  noted 
in  Section  may  well  be  nonlinear  and/or  time- varying,  depending  on  the  statistics 

of  the  signal  and  noise  present.  Kalman  (Ref.  4o)  remarks  that  as  far  as  he  is 
aware,  it  is  not  known  what  is  the  most  general  random  signal  and  noise  for  which 
the  conditional  distribution  function  satisfies  conditions  1^2,  and  3  above. 

Special  Cases 

f(€)  ^  6^.  It  was  shown  in  Section  C  that  when  f (e)  =  g^,  Eq  37  for  the 
optimum  system  minimizes  E(€2)  without  the  requirement  that  p(c^/?‘)  satisfy  condi¬ 
tions  1  ,  2,  and  3  (following  Eq  36).  This  was  also  noted  by  Kalman  in  Ref.  4o. 
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Gaussian  Signal  and  Noise >  In  the  important  case  of  nonstationary  Gaussian 
signal  and  noise ^  the  conditional  distribution  of  c^  given  'r"  is  Gaussian  (as 
shown  by  Cramer^  Ref.  37^  p.  515)^  which  satisfies  conditions  2^  and  3-  As 
demonstrated  in  Section  the  output  of  the  optimum  system  (Eq  26)  is  a  linear 
function  of  the  past  values  of  the  sum  of  signal  and  noise-  If  the  signal  and 
noise  are  stationary  as  well  as  Gaussian^  the  absolute  optimum  system  for  all 
f (e)  as  defined  in  Eq  26  is  the  Wiener  system. 


Changes  Extension  of  Sherman *s  Results  for  Gaussian  Processes.  In  many  situa¬ 
tions  it  is  desirable  that  the  mean  error^  be  equal  to  zero.  For  these  cases 
Chang  (Ref.  10)  has  shown  that^  for  Gaussian  processes^  minimizing  0^  also  mini¬ 
mizes  Sherman*s  f(e).  This  will  be  true  even  if  the  symmetry  property  of  f(G) 
in  Eq  3^  is  relaxed.  The  above  result  is  almost  intuitively  obvious^  since  a 
Gaussian  process  is  determined  by  its  mean  and  variance^  and  with  zero  mean 
only  the  variance  can  be  reduced  to  reduce  f(€). 


Kaufman *s  (Ref.  16)  Performance  Measure.  Kaufman  has  used  as  a  performance 
measure 


X 


lim  _L  V 


2n. 


'dt 


K 

E 

n=1 


CnE 


2n 


(58) 


for  cases  where  is  not  sufficiently  meaningful.  He  restricted  his  analysis 
to  the  following  situations : 

1  .  signal  and  noise  are  statistically  independent  processes 
with  non- Gaussian  probability  density  functions  which 
are  symmetric  about  the  mean 

2.  only  linear  systems  are  considered. 

Kaufman  showed  how  to  evaluate  and  minimize  X  analytically.  The  procedure  was 
approximate  and  quite  involved^  and  only  calculations  of  ^ere  demon¬ 

strated. 

The  system  that  yields  the  absolute  minimum  X  depends  on  the  conditional 
density  function  of  the  signal^  given  the  past  history  of  the  sum  of  signal  and 
noise  (as  shown  by  Eq  37 )  •  The  information  given  in  Item  1  above  is  not  suffi¬ 
cient  to  apply  Sherman *s  simplification  to  this  perfoimance  measure.  But  if  the 
conditional  density  function  of  c^  given  r"  satisfies  Sherman's  conditions^  then 
f(€)  may  as  well  be  replaced  by  The  absolute  minimum  system  quite  probably 

will  not  be  linear  since  the  signal  and  noise  are  non-Gaussian. 
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G.  PUGACHEV'S  ATO  STREETS'  MINIMIZATION  OF  GENERAL  F(e)  CRITERIA 

WHEN  THE  SIGNAL  IS  PARTIALLY  DETERMINISTIC 

In  this  section,  optimization  procedures  for  a  class  of  inputs  more  general 
than  that  hitherto  considered  will  he  summarized.  These  inputs  are  allowed  to 
have  partially  deterministic  components.  This  corresponds  to  an  important  group 
of  flight  control  inputs  where  the  input  may  have  a  component  of  known  amplitude 
and  form  (as  in  bang-bang  control),  but  where  the  timing  between  successive 
applications  of  this  input  component  is  random.  As  described  in  Section  E  of 
this  chapter,  the  minimum  system  also  minimizes  a  wide  class  of  other  F(g) 
measures.  Pugachev  (Ref.  l4)  has  shown  that,  when  the  input  contains  Gaussian 
signals  plus  a  partially  deterministic  component,  the  minimiam  system  is  either 
equal  to,  or  very  simply  related  to,  the  minimum  F(e)  system,  where  F(€)  is  a 
general  loss  fimction.  Pugachev's  results  will  now  be  briefly  summarized  and  a 
brief  account  given  of  their  application  by  Streets (Ref.  4l) .  This  will  be  fol¬ 
lowed  by  a  summary  of  Lubbock's  procedure  (Ref.  42  )  for  optimizing  a  class  of 
nonlinear  systems  to  yield  minimum  with  non-Gaussian  inputs.  The  reader 
concerned  only  with  random  Gaussian  inputs  will  find  little  use  for  the  analyses 
of  Pugachev,  Streets,  and  Lubbock,  and  he  is  advised  to  skip  these  sections  and 
turn  to  the  Summary  on  page  59* 

Pugachev  (Ref.  l4)  considers  an  input  signal  given  by 


N 

r(t)  =  s(t)  +  n(t)  +  °'k^k(‘*^) 

k=1 


(39) 


where  s(t) 
n(t) 


stationary  Gaussian  signal 
stationary  Gaussian  noise 


a^fjj.(t)  =  additional  signal  tenns;  the  are 

known  functions  of  time^  and  the  a-jj.(t) 
are  random  variables  with  a  known 
Gaussian  joint  probability  density 
function^  *"• 


It  is  also  assumed  that  s(t)  and  n(t)  both  have  zero  means^  known  auto-  and  cross¬ 
correlation  functions^  and  are  statistically  independent  of  the  additional  signal 
terms.  The  desired  output^  c^(t)_,  is  a  linear  function  of  the  total  signal^ 

S(t)  +  CCr]£f]j^(t)  . 


5^ 


Piigachev  shows  that  for  an  arbitrary  error  criterion,  F(€)  {[not  restricted 
to  be  a  nondecreasing  even  function  of  (e)[],  the  optimum  system  is  the 
optimum  linear  system  plus  a  constant  bias  term,  k,  as  shown  in  Fig.  9* 


optimuiii 
linear  system 


k(t)  =  k 


c 


Figure  9-  Pugachev’s  Optimum  System 

The  error  has  a  mean  value  of  k  and  changing  the  form  of  F(g)  changes  k,  but 
does  not  affect  hx(t),  the  optimum  system  weighting  function-  If  F(c)  =  f(€), 
an  even  function  of  the  class  defined  by  Sherman,  then  k  is  zero. 

Streets  (Ref.  l^-l)  has  worked  out  two  problems  of  the  type  considered  by 
Pugachev.  He  demonstrated  that  a  theoretical  (linear,  time-varying)  system  can 
be  designed  which,  by  taking  advantage  of  the  partially  deterministic  character 
of  the  input,  yields  a  lower  than  a  Wiener  system.  The  problems  studied  have 
Gaussian  white  noise  for  n(t),  while  s(t),  the  purely  random  signal  component, 
is  zero  in  Eq  39.  Streets  described  the  partially  deterministic  signal  as 
follows.  The  signal  is  a  stationary  process  which  consists  of  a  sequence  of 
random  steps  random  ramps  in  the  second  problem]  of  constant-amplitude  [velocit^^ 
segments  (see  Fig.  10  ).  The  amplitude  changes  at  "event  points"  that  are  Poisson 
distributed  in  time.  The  amplitudes  have  a  Gaussian  distribution  with  zero  mean. 
The  amplitudes  before  and  after  an  event  point  are  independent.  The  desired  out¬ 
put  equals  the  signal. 
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The  signal  described  above  has  a  power  spectrum  of  the  form  — (Ref.  *50, 

0^  +  v2  ^  ^ 

P*  ^57)  •  li*  the  noise  has  a  white  spectrum  “  constanl^  ^  the  Wiener  system 

is  a  first-order  filter.  In  competition  with  this^  Streets*  theoretical  optimum 
system  requires  the  following  three  components: 


1 .  a  perfect  event-point  detector 

2.  an  adaptive  device  which  monitors  the  time  from  the  last  event 
point  and  adjusts  the  system  parameters  accordingly.  At  each 
event  point  this  device  sets  the  output  signal  equal  to  the 
expected  mean  and  other  output  initial  conditions  equal  zero 

5.  a  linear^  time-varying  filter  plus  a  bias  term  when  using  non¬ 
even  criteria. 


For  low-frequency  signal- to-noise  ratios  of  1  and  100^  the  improvement  in 
performance  given  by  Streets*  theoretical  system  is  such  that  the  mean  square 
errors  are  reduced  to  90  percent  and  55  percent  (respectively)  of  the  Wiener 
values.  Similar  results  were  obtained  for  the  second  problem  using  random  ramps. 

Streets  remarks  that  the  improvement  in  performance  given  by  the  theoretical 
system  over  the  Wiener  system  is  not  as  great  as  might  be  expected-  This  is 
because  the  input  signal  in  each  example  is  a  "near  Gaussian"  signal.  A  more 
non- Gaussian  signal  would  be  expected  to  offer  a  larger  potential  improvement 
through  use  of  a  nonlinear  or  adaptive  system. 

It  is  important  to  note  the  following  points  about  Streets  *  theoretical  system 

1  .  it  is  optimum  for  the  criterion  of  Pugachev 

2.  no  other  adaptive  or  nonlinear  system  can  give  a  smaller 

3-  it  can  be  analytically  designed  and  is  physically  realizable 
except  for  the  perfect  event-point  detector. 

Streets  assumed  a  perfect  event-point  detector  to  simplify  the  analysis  and 
to  delineate  the  absolute  lower  bound  on  the  mean  square  error.  If  a  Wiener 
system  is  close  to  this  lower  bounds  there  is  little  need  to  attempt  to  approxi¬ 
mate  the  Streets  theoretical  optimum  system^  since  any  physically  realizable 
event-point  detector  must  operate  in  a  noisy  environment  and  may  miss  event  points 
or  have  a  delay  in  detecting  them.  Streets  states  that  both  these  conditions 
will  seriously  degrade  the  performance  of  the  time-varying  system^  but  of  course 
will  not  affect  the  performance  of  the  Wiener  system.  Streets  also  states  that 
if  the  idealized  signal  is  a  poor  model  of  the  actual  signal^  the  performance  of 
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the  theoretical  system  is  degraded  to  a  much  larger  extent  than  that  of  the 
Wiener  system^  since  the  Wiener  system  does  not  depend  on  the  detection  of 
event  points. 

H.  LUBBOCK'S  PROCEDURE  FOR  SYNTHES IS_^F  A  CLASS  OF 

WONLIHEAR  SYSTEMS  HAVING  MINIMUM  eS 

Lubbock  (Ref.  42)  showed  how  to  minimize  the  mean  square  error  for  a  special 
class  of  nonlinear  systems  subjected  to  stationary  random  inputs  vith  a  general 
(i.e.^  not  necessarily  Gaussian)  distribution.  The  class  of  nonlinear  systems 
is  that  which  can  be  represented  by  a  nonlinear  gain  operating  upon  the  input 
followed  by  a  linear  system  (see  Fig.  ll). 


’(t) 


:(t) 


m 


Figure  1 1 .  Nonlinear  Gain  Followed  by  a  Linear  System 


Lubbock's  procedure  is  directly  of  interest  in  the  context  of  this  report  as  an 
example  of  how  a  fixed-form  nonlinear  system  may  be  optimized  for  minimum  mean 
square  error.  This  could  be  advantageous  for  flight  control  systems  subject  to 
markedly  non-Gaussian  inputs. 

The  input  and  output  are  related  by 


c(t) 


t)3  h(T)dT 


(4i) 


This  is  a  special  case  of  the  following  equation^  which  defines  a  class  of 
filters,  T)-!  : 


c(t) 


dr 


(42) 


Lubbock  assumes  that  the  kernel  fiinction,  K(  r,  t)  ,  can  be  expanded  in  terms  of  a 
set  of  orthogonal  functions^  ®a^^^  (which  are  polynomials  in  x  of  degree  a) .  The 
output  then  becomes 
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c(t) 


CD  r  CO 

=  23  I  ■  '^)1 

a=0j0 

■which  can  he  synthesized  hy  a  multipath  filter  as  shown  in  Fig.  12. 


Figure  12.  Lubbock’s  Optimum  Multipath  System 

Lubbock  claims  that  by  using  a  finite  number  of  terms  in  the  expansion^  the 
filter  can  be  physically  realized  and  made  to  approximate  any  system  defined 
by  Eq  42  with  ary  desired  accuracy  (assuming  that  the  expansion  converges). 

An  integral  equation  for  the  optimum  filter  of  class  is  derived  by  varia¬ 
tional  calculus-  The  information  required  to  solve  this  equation  is  the  joint 
probability  density  of  the  inputs  P2(^i  ;  ^2^  ^  joint  probability 

density^  p^Cr^  ^  c^^  ^  between  the  input  and  the  desired  output-  Lubbock 

proceeds  to  foiina3_ly  expand  each  density  function  in  a  double  series  involving 
the  first  probability  density  fimctions,  p(^)  and  p(c^^);  polynomials^  0g^(r)^ 
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which  are  orthonormal  with  respect  to  p(r)  and  p(C(i);  and  the  cross-correlation 
between  the  outputs  of  the  polynomials  (see  Fig.  lO).  The  restrictions  on  the 
joint  probability  density  function  required  to  validate  this  procedure  are  not 
enimierated  by  Lubbock,  nor  are  they  known  to  the  present  authors.  Using  the 
truncated  series  representation  for  the  joint  probability  density  functions,  the 
integral  equation  for  the  optimum  system  becomes  a  finite  number  of  simultaneous 
integral  equations. 

Lubbock  notes  that  there  are  situations  in  which  the  optimum  general  system  of 
class  reduces  to  the  Wiener  system.  The  conditions  for  this  simplification  to 
take  place  define  a  class  of  joint  probability  density  functions  which  includes 
Gaussian  density  functions. 

I.  SUMMARY 

This  chapter  has  been  concerned  with  random  input  performance  measures  that 
are  expressible  as  functions  of  error  alone,  the  error  being  defined  as  the  dif¬ 
ference  between  the  actual  output  and  some  specified  or  desired  output.  The  most 
important  result  presented  has  been  the  demonstration  that  for  a  veiy  wide  class 
of  such  f(e)  loss  functions,  f(e)  is  minimized  by  the  Wiener  or  Phillips  system, 
provided  that  the  input  is  Gaussian.  Many  flight  control  inputs  have  approxi¬ 
mately  Gaussian  distributions  (e.g.,  atmospheric  turbulence,  see  Ref.  and  . 
Hence,  for  many  situations  of  practical  interest  in  flight  control  optimization,  the 
optimum  system  is  the  same  for  minimxim  |e|,  e^,  etc.,  and  changing  the  loss 
function  accomplishes  no  real  change  in  the  optimum  system.  Since  it  is  so  diffi¬ 
cult  to  "escape"  from  the  minimum  criterion,  it  is  worth  reiterating  its 
advantages  and  deficiencies.  As  noted  in  the  Introduction,  the  principal  require¬ 
ments  for  a  perfonnance  criterion  for  flight  control  systems  are  validity, 
selectivity,  and  ease  of  application.  The  minimum  criterion  may  be  briefly 
assessed  in  terms  of  these  qvialities  as  follows : 

Validity .  Minimum  e2  ^3  only  moderately  valid  as  a  performance  criterion, 
because  it  frequently  selects  damping  ratios  somewhat  lower  than  would  be  con¬ 
sidered  optimum  on  the  basis  of  transient  response.  Figure  5  illustrates  this 
point:  for  an  integrated  white  noise  input,  the  second-order  system  considered 
gave  minimimi  at  ^  =  0.5* 
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Selectivity >  In  all  the  cases  that  have  been  examined  in  the  course  of 
preparing  this  report,  the  optimm  system  yielded  a  mean  square  error  only  a 
little  less  than  that  given  by  a  wide  range  of  off-optimum  systems.  For  the 
\init  numerator  second-order  system  of  Fig.  5,  varying  ^  between  0,4  and  0.6 
raised  only  2.5  percent  above  the  minimum.  A  similar  lack  of  selectivity  is 
exhibited  by  lE^  for  transient  inputs  (see  Fig.  l6  of  Ref.  l).  (By  comparison, 
the  degree  of  selectivity  displayed  by  the  ITAE  performance  measure  for  step 
inputs  is  generally  regarded  as  acceptable  in  this  respect,  and  this  gives  a 
7.5  percent  increase  in  the  measure  for  a  change  of  ^  of  ±0-1  from  the  optimum 
of  ^  =  0,76  for  the  above  system.) 

Ease  of  Application.  The  present  study  is  directed  principally  at  "pencil 
and  paper"  methods  of  optimization,  and  it  is  essential  that  any  criterion 
selected  possess  analytic  forms  which  are  simple  enough  for  manual  calculations. 

It  is  fortunate  in  this  regard  that  (as  noted  in  the  Introduction)  flight  control 
systems  can  almost  always  be  represented  by  low-order  "equivalent  systems."  This 
simplification  keeps  Phillips- type  optimization  within  the  bounds  of  practicality. 
For  high-order  systems  and  input  spectra,  the  tabulated  Phillips  integral  forms 
for  can  become  very  lengthy.  However,  none  of  the  examples  in  this  report 
required  more  than  a  few  man-hours  for  the  evaluation  of  these  integral  forms. 

The  criterion  is  thus  judged  to  be  adequately  easy  to  apply  to  linear  systems. 
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CHAPTER  II 


EXCEEDARCE  CRITERIA  ARD  CRITERIA  FOR  MIXED  RARDOM  ARD  DETERMIRISTIC  IRPUTS 

Section  A  of  this  chapter  discusses  exceedance  criteria^  i.e.,  criteria 
defined  in  terms  of  a  certain  fixed  level  of  the  error  or  output.  Usually  one 
attempts  to  minimize  the  average  number  of  times  per  second  that  the  specified 
level  is  crossed.  Alternatively,  one  may  seek  to  minimize  the  probability  that 
the  error  will  exceed  the  specified  level;  such  a  "probability  criterion"  was  pro¬ 
posed  by  Zadeh  and  Ragazzini  in  Ref.  l8  and  is  examined  in  Section  B  of 
this  chapter.  Zadeh  and  Ragazzini  considered  inputs  comprised  of  a  deterministic 
signal  plus  random  noise.  It  will  be  shown  that  for  such  inputs  the  probability 
criterion  is  of  little  value ,  although  mean  square  error  criteria  can  be  success¬ 
fully  applied.  A  discussion  of  this  last  point  is  given  in  Section  C. 

A.  EXCEEDARCES 

Thorson  and  Bohne  (Ref.  ^5),  among  others  (Ref.  k6  and  ^7) ,  have  discussed 
methods  of  calculating  the  expected  total  number  of  exceedances  of  a  prescribed 
level  experienced  by  a  specified  vehicle  during  a  given  mission.  This  number  of 
exceedances,  coupled  with  information  on  the  fatigue  characteristics  of  the  struc¬ 
ture,  can  be  used  to  determine  the  expected  life  of  the  vehicle.  Total  exceedance 
numbers  differ  from  the  performance  measures  previously  discussed  in  that  they  are 
not  primarily  intended  to  assess  dynamic  performance,  but  instead  describe  another 
fundamental  parameter  of  system  effectiveness,  i.e.,  the  vehicle's  expected  life. 
However,  it  is  logical  to  inquire  whether  exceedance  concepts  can  be  used  to  form 
dynamic  performance  meas\ires .  In  order  to  answer  this  question,  some  specific 
formulas  for  exceedances  are  required,  and  these  will  now  be  presented. 

Exceedances  can  be  expressed  in  two  forms  (which  are  frequently  equivalent) : 

1  .  the  average  number  of  exceedances  of  a  given  level  by  the 
output  (or  error  quantity)  per  unit  time  (or  per  single 
mission  for  fatigue  calculations  to  determine  the  total 
permissible  number  of  missions) 

2.  the  fraction  of  the  total  time  during  which  the  output  (or 
error  qiiantity)  exceeds  the  given  level. 
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For  stationary  random  inputs^  the  ergodic  hypothesis  permits  form  2  to  be  expressed 

Q  S 

5.  the  probability  that  the  output  or  error  quantity  will 
exceed  the  given  level. 

In  Ref.  l8  this  last  form  is  proposed  for  the  optimization  of  systems  subjected 
to  inputs  having  both  random  and  deterministic  components.  This  probability 
criterion  is  examined  in  Section  B  of  this  chapter^  and  is  shown  to  be  difficult 
to  apply  in  nontrivial  cases.  Throughout  the  section  that  now  follows^  station¬ 
ary  random  inputs  will  be  assumed. 

Formulas  for  Average  Exceedances  Per  Second 


Rice  (Ref.  17)  has  shown  that  for  a  Gaussian  signal^  v(t) ^  with  zero  mean, 
the  expected  number  of  axis  crossings  per  second  (including  crossings  with  both 
positive  and  negative  slopes)  is 
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where  f  =  77-  =  frequency 
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For  rectangular  spectra  ranging  from  (%  to  the  expected  number  of  axis 
crossings  per  second  is 
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When  =  0  this  reduces  to  Hence  for  many  spectra  of  practical  interest 

the  number  of  zero  crossings,  Nq,  can  serve  as  an  indication  of  bandwidth.  Nq 
times  the  total  operating  time  may  also  be  employed  in  a  fatigue  criterion,  since 
it  indicates  the  total  expected  niomber  of  stress  reversals.  In  situations  where 
knowledge  of  Nq  is  significant,  the  average  number  of  maxima  per  second,  %,  may 
also  be  important.  This  is  given  by  Rice  as: 
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For  rectangular  spectra  of  the  type  considered  in  the  previous  paragraph^  this 
reduces  to 
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When  (X^  =  0^  the  average  number  of  maxima  per  second  is  simply  — 

Of  much  more  direct  interest  as  a  fatigue  criterion  and  a  possible  performance 
measure  is  the  average  number  of  exceedances  per  second  of  some  specified  output 
level,  L-  For  Gaussian  inputs  this  is  given  by  Rice  (Ref.  17)  as 


-l2/2o2 

Nl  =  — 2 -  ^  %  '  (^7) 

where  Nq  =  average  niJinber 
of  zero  crossings  per  second 

Strictly,  this  formula  only  applies  when  L  »  a.  The  accuracy  of  the  formula  for 
smaller  values  of  L  has  been  investigated  by  Press,  Meadows,  and  Hadlock  in  Ref.  43. 
They  conclude  that  for  L/u  >  2,  the  formula  is  in  most  cases  valid.  For  values 
of  L/u  <  2,  it  tends  to  slightly  underestimate  the  number  of  exceedances.  It  is 
noted  in  Ref.  43  that  for  moderately  flexible  airplanes,  the  formula  is  10  to  1 5 
percent  low  at  L/a  =  1  and  2  to  3  percent  low  at  L/ a  =  2.  This  would  certainly 
be  acceptable  for  purposes  of  systems  optimization;  hence  Rice's  formulas  will  be 
retained  in  the  discussion  that  now  follows. 

Exceedance  Rumbers  as  Performance  Criteria 

The  conventional  use  of  exceedances  as  fatigue  criteria  relates  to  the  number 
of  output  exceedances  above  a  given  level.  To  form  dynamic  performance  criteria 
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it  would  seem  to  be  more  appropriate  to  consider  the  number  of  error  exceedances. 
However^  there  is  some  interest  in  determining  whether  conventional  output 
exceedance  fatigue  measures  can  also  be  used  as  performance  measures.  Therefore, 
in  this  section  both  output  and  error  exceedances  will  be  investigated  for  a 
second-order  unit  numerator  system  subjected  to  a  stationary  Gaussian  input,  and 
the  variation  of  these  exceedance  numbers  with  ^  will  be  examined.  The  principal 
results  obtained  are 

1 .  minimizing  the  output  (or  error)  exceedances  is  not  equivalent  to 
minimizing  the  mean  square  output  (or  error) 

2.  the  direct  application  of  error  exceedance  formulas  to  some  input 

environments  is  hampered  by  the  failure  of  certain  integrals  to 
converge-  Apart  from  this,  the  criterion  is  fairly  easy  to  apply, 
and  is  valid  and  selective  when  the  low-frequency  ratio  of  signal 
to  noise  power  spectra  is  fairl  high  (50:1 ).  Unfortunately,  for 
lower  values  of  this  ratio  the  number  of  error  exceedances 

tends  to  be  minimized  by  excessively  high  ^  for  the  second-order 
system  investigated 

3-  further  investigation  of  error  exceedance  criteria  is  recommended  to 
overcome  the  convergence  problem  and  to  explore  a  si;iggestion  (Ref.  57) 
that  they  are  used  by  hxoman  pilots  performing  tracking  tasks. 

Output  Exceedances 

An  example  will  now  be  given  to  illustrate  point  1  above.  The  system  consid¬ 
ered  has  the  transfer  function 

H(s)  =  ^ ^ -  (48) 

s^  +  2^s  +  1 

The  input  power  spectrum  is 

4>_c,(od)  =  4>  (cd)  =  — 7^  (49) 

rr^'  2.2 

a  +0) 

and  zero  noise  is  assumed.  Combining  Eq  48  and  49,  the  output  power  spectrum  is 
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In  order  to  evaluate  Nq,  the  niimber  of  axis  crossings^  it  is  necessary  to  compute 
/q  and  (S)(a))dcjD  in  Eq.  44.  These  integrals  may  he  evaluated  hy  use  of 

the  Phillips  integral  formulas.  The  final  result  is 
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The  average  number  of  exceedances  per  second  of  an  output  level  L  is 


- L^2a ( 1  +  2aC  +  C  ^ ) 
a  ^  (2t  +  a) 

2m(2^  +  a)  '  ^ 


(52) 


Figures  1 3  and  1 4  illustrate  how  the  number  of  exceedances  varies  with  ^  for 
a  ~  0.1 ^  and  for  L  =  1 ,  3^  5^  and  7*  As  would  be  expected^  the  exceedances 
decrease  monotonically  with  Increasing  this  is  physically  reasonable  since 
increasing  ^  implies  that  the  system  is  becoming  more  sluggish  less  liable 
to  overshoot. 

To  obtain  a  physical  "feel"  for  the  meaning  of  this  result^  consider  Fig.  15. 

This  figure  illustrates  the  variation  with  ^  of  0^  ,  ,  ^  the  mean  sc^uare  output. 

^2  f  ounput) 

(output  evaluated  by  computing  (s)ds  using  Phillips  integrals.)  Com¬ 

paring  Fig.  15  with  Fig.  13  and  l4^  it  is  seen  that  the  fall-off  of  Nl  with 
increasing  ^  is  more  rapid  than  the  fall-off  of  This  is  made  clear  in 

Fig.  16  which  shows  the  ratio  Nl/c  plotted  versus  ^  for  L  =  ^  ,  3,  5^  and  7.  For 
each  of  these  values  of  L,  Increasing  ^  causes  to  diminish  more  rapidly  than  a 

as  shown  by  the  monotonic  decrease  of  the  ratio  WL/a-  Figures  1 3,  l4^  and  15 
show  that  minimizing  either  o^^tput  yields  ^  -*-co.  Minimization  of  these 

output  quantities  thus  appears  unpromising  as  a  performance  criterion  and  will  not 
be  puxsued  further  in  this  report. 

Error  Exceedances 


The  unit  numerator  second-order  system  will  now  be  used  to  investigate  error 
exceedances  as  performance  measures.  Consider  the  system  illustrated  in  Fig.  17 
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Figure  1  J.  Output  Exceedances  W-i  and  Wj  vs  ^  for  a  Second-Order  Unit  Numerator  System 
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Output  Exceedances  and  Ny  vs  ^  for  a  Second-Order  Unit  Numerator  System 
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Figure  15*  Mean  Square  Output  vs  ^  for  a  Second-Order  Unit  Numerator  System 
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Figure  l6.  Ratio  of  Output  Exceedances  to  for 
Second-Order  Unit  Numerator  System 
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=  0.1 


with  statistically  independent  signal  and  noise  input: 


Figure  IT-  System  for  Investigation  of  Error  Exceed 

The  error  is  the  difference  between  the  signal  and  the  outxjut. 
of  exceedances  per  second  of  some  specified  level  of  the  error. 


artces 

The  average  number 
L,  is 


_  i 


(53) 


where  =  the  average  number  of  zero 

crossings  per  second  of  the  error 


N 


^0 


1^ 


’  1  o 

I  .  (s)ds 

2it,T  J-joo  eg 

—  I  $  (s)ds 
J-jco  ee 


1  /2 


S=j(X) 


Using  the  formula  for  the  error  power  spectrum,  gg(s) ,  given  in  Ref.  ^0,  p.  239^ 


%Js)  =  [l  -  H(s)]  [l  -  H(-s)]<&^^(s)  +  H(s)H(-s)<J>^^(s) 
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and  substituting  the  expressions  given  in  Fig.  17  for 


and  yields 


-s(-s  +2^) 

s(s  +  2U 

_(s^  -  2^s  +  1)_ 

_( +  2^  s  +  1  ) 

(-S  +  a)^(s  +  a)^ 

(s^  -  2^s  +  l)(s^  +  2^s  +  1) 


(5^^) 


Note  that  if  the  denominator  of  the  input  pover  spectrum  had  been  only  of  second 
order  (as  in  the  output  exceedance  example)^  the  numerator  of  Eq  53  for  N^q  would 
have  failed  to  converge  since  would  not  tend  to  zero  for  large  o).  This 

disadvantage  is  more  likely  to  occur  for  the  error  exceedance  formulas  than  for 
the  output  exceedance  formulas  because  the  order  of  the  numerator  of  ^>^^(00)  in 
Eq  55  is  higher  than  the  order  of  the  numerator  of  the  corresponding  ^^^(03) - 
This  lack  of  convergence  does  not  appear  to  reflect  any  physical  peculiarity  of 
the  actual  system  response^  and  it  should  be  possible  to  recast  the  formulas  to 
avoid  this  difficulty.  This  problem  may  merit  further  investigation. 

Evaliiating  N^^  using 


yields 


,2  _ 


/<^(a  +  a^C  +  +  4^^) 


and 


N, 


^0 


»] 


(8a5  +  4a  +  4a5)^  +  (l6a^  + 


1  /2 


(55) 


(56) 


where  Z  = 


/c^2a^[a  +  a^(a^  +  4)^  +  4a(a^  +  +  4a^^5^  ^^2 

^ w 

(8a^  +  4a  +  +  (l6a^  +  l6a^)^^  + 
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Figure  l8.  Error  Exceedances  and  Mean  Square  Error  vs  ^  for 

a  Second-Order  Unit  Numerator  System  with  Low- 
Frequency  Signal-to-Noise  Ratio 
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C 

Figure  19*  Error  Exceedances  and  Mean  Square  Error  vs  ^  for 
a  Second-Order  Unit  Numerator  System  with  50:1 
Low-Frequency  Signal-to-Noise  Ratio 
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Figure  18  illustrates  a  typical  variation  of  and  with  ^  for  L  =  2^ 
a  =  0.25^  N  =  1^  and  =  1 /50*  this  situation  the  low-frequency  ratio  of 

the  signal  to  noise  spectra  is  approximately  5-1  •  With  this  signal  to  noise 
ratio,  neither  nor  yi^l^  a  satisfactory  criterion. 

Figure  19  illustrates  the  effect  of  changing  to  1 /5^  thus  making  the  low- 
frequency  signal  to  noise  spectra  ratio  50:1 •  There  is  a  marked  change  in  the 
behavior  of  the  graphs  which  now  display  highly  selective  minima  at  ^  =  0.8 
to  0.9  for  large  values  of  L. 

This  change  in  behavior  makes  it  difficult  to  form  a  complete  assessment  of 
error  exceedance  criteria  without  a  much  larger  number  of  examples. 

B.  PROBABILITY  CRITERIA 

Ragazzini  and  Zadeh  (Ref.  1 8)  have  proposed  the  probability  criterion 

maximum  p  =  maximum  probability  je]  <  L 

for  handling  situations  where  it  is  desirable  that  the  magnitude  of  the  error  be 
less  than  a  certain  critical  value  L;  i.e.,  all  errors  larger  than  L  are  equally 
bad^  while  those  smaller  are  equally  acceptable.  This  may  well  be  the  case  in 
some  problems  of  ballistics  and  many  flight  control  applications.  It  is  apparent 
that  for  such  cases  the  mean  square  error  does  not  correctly  reflect  the  require¬ 
ments  of  the  system.  Ragazzini  and  Zadeh  *s  development  of  the  probability 
criterion  is  summarized  below;  it  will  be  shown  that  in  nontrivial  cases,  the 
criterion  is  difficult  to  apply. 

The  problem  is  set  up  by  Ragazzini  and  Zadeh  in  the  following  way: 


r  + 


G(s) 

c 

r 

H(s) 

Figure  20.  Block  Diagram  for  Probability  Criterion  Optimization 
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The  error  arises  from  a  transient  desired  input,  and  from  random  disturbances 
arising  both  inside  and  outside  the  system;  i.e., 

^  =  ^s  ^n  (57) 

where  €g  is  the  systematic  error  due  to  transient  input 

is  stationary  random  noise  (normally  distributed) 

Note  that  this  error  is  the  conventional  "actuating 
error,"  not  the  difference  between  input  desired 
output 

The  stiaicture  and  components  of  the  system  are  specified  completely  except 
for  a  number  of  adjustable  parameters,  ,  0C2,  oc^.  The  objective  is  to 
assign  values  to  which  maximize  the  probability  that  at  a  prescribed  instant, 
t  =  to,  the  magnitude  of  e  will  be  less  than  a  given  tolerance,  L.  With  a 
normally  distributed  noise  variable,  the  probability,  p,  may  be  written 


p  =  probability 


where  CgQ  is  the  value  of  Sg  at  t  =  tQ 


is  the  variance  of  the  noise  =  E(eQ) 
L  is  the  critical  value  of  [el 


If  the  sequence  of  events  under  consideration  is  repeated  many  times  (with  the 
same  input,  the  same  initial  conditions,  and  the  same  free  parameters),  the  value 
of  Sgo  will  be  constant.  With  this  understanding,  the  ensemble  average  mean 
square  error  takes  the  form 


(59) 


In  Ref.  l8,  p  is  compared  with  E  [e^(tQ  )]  to  examine  whether  maximizing  p  also 
imizes  .  From  Eq  59,  for  minimum  E[e2(to)J  ,  it  is  necessary  that 


0,  i  =  1  ,  2,  •  •  •  n^ 


(60) 
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Substituting  Eq  59  into  Eq  60  yields 


^sO 

sO  ^ 


0 


(61) 


Similarly^  for  p  to  be  a  maximuin^  it  is  necessary  that 

Sp  _  5p  / ^^so\  ^  Sp  /  Sa  \ 
”  ScgQ  \  /  5a  \5ai/ 

where  i  =  1  ^  2^  •  •  •  n 


(62) 


Choosing  the  coefficients  Sp/Scgo  5p/5a  in  Eq  62  to  exactly  match  those  in 
Eq  6l  demands  that  the  ratios  €so/(5p/5€go)  n/(5p/5a)  should  be  equal. 

Ragazzini  and  Zadeh  show  that  this  requirement  can  be  satisfied  approximately 
over  a  wide  range  of  values  of  Ggo^  when 


Usol  «  ^ 

and 


or 


and 


I  Sol  »  ^ 


=  L 


It  is  concluded  in  Ref.  l8  that  under  these  conditions^  maximizing  p  is 
the  same  as  minimizing  E  However^  in  all  practical  situations,  it  will 

be  found  impossible  to  both  satisfy  these  conditions  and  achieve  a  nontrivial 
result.  Either  g^q  changes  with  changes  in  the  oc^*s,  or  it  does  not.  If  it  does 
not,  all  one  can  do  to  maximize  p  is  to  minimize  o^.  If  g^q  changes  with  the 
o^’s,  there  is  no  guarantee  that  the  inequality  conditions  above  will  always 
hold  as  the  (Xj_*s  are  varied.  Because  this  is  the  most  likely  case,  trial  and 
error  methods  must  be  resorted  to  in  order  to  use  the  maximum  p  criterion.  Thus, 
this  criterion  fails  to  meet  the  requirement  of  ease  of  application. 
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c.  A  NOTE  ON  THE  MINIMUM  MEAN  SQUARE  ERROR  CRITERION 

WITH  MIXED  DETERMINISTIC  AND  RANDOM  INPUT  COMPONENTS 

The  previous  section  described  how  probability  criteria  could  be  applied  to 
a  system  subjected  to  a  deterministic  command  corrupted  by  random  noise.  Zadeh 
and  Ragazzini  (Ref.  12)  have  also  considered  the  case  where  the  total  signal  has 
three  components: 

1 .  n(t),  stationary  random  noise  with  zero  mean 

‘ 

2.  iii(t)^  a  stationary  random  signal  with  zero  mean 

3*  a  deterministic  signal  representable  as  a 

polynomial  in  t  with  a  finite  number  of  terms - 

For  this  case  the  minimum  mean  square  error  criterion  is  used  with  a  finite 
observation  time^  T;  i-e.^  the  task  is  to  find  the  system  that  will  minimize  the 
ensemble  average  (over  a  large  number  of  trials)  of  the  mean  square  error^  the 
mean  of  each  trial  being  taken  over  the  observation  time^  T. 

An  accoxint  of  Zadeh  and  Ragazzini  *  s  procedure  occupies  the  whole  of  Chapter  8 
of  Ref.  7  i  "the  procedure  is  lengthy  to  explain^  complicated^,  and  demands  consid¬ 
erable  computational  labor  for  problems  of  practical  interest-  Indeed^  both 
Ref.  7  and  12  give  only  simplified  examples  for  which  either  the  random  or  the 
deterministic  component  of  the  signal  is  absent.  From  the  viewpoint  of  the 
present  study — which  is  directed  toward  "pencil  and  paper"  methods  of  optimiza¬ 
tion  — the  method  fails  to  meet  the  requirement  for  ease  of  application.  In  view 
of  this y  and  because  it  is  difficult  to  present  a  detailed  account  of  Zadeh  and 
Ragazzini ’s  procedure  that  is  appreciably  shorter  than  that  given  in  Ref.  7, 
only  an  outline  of  the  method  is  given  here. 

The  presence  of  the  detemninistic  component  in  the  signal  leads  to  the 
establishment  of  certain  conditions  upon  the  time  moments  of  the  optimum  impulsive 
response.  The  minimization  of  the  mean  square  error  is  then  effected  by  combining 
these  conditions  using  Lagrange  multipliers  in  a  fashion  similar  to  the  minimiza¬ 
tion  of  performance  criteria  of  systems  subject  to  constraints  due  to  nonlinear- 
ities^  etc.  (e.g.^  Ref.  6)* 

Assiaming  that  signal  and  noise  data  are  available  only  in  the  interval  0  to  T^ 
the  system  response  is  given  by 
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/•T 

c(t)  =  I  |g(t  "  t)  +  m(t  -  t)  +  n(t  -  t)|  h(T)dT  (65) 

JO 

where  h(T)  is  the  actual  system  weighting  function 

Since  g(T)^  by  definition^  is  expressible  as  a  polynomial  in  t  with  a  finite 
number  of  terms  ^  it  can  be  expanded  in  a  Taylor  series  as 

g(t  -  t)  =  g(t)  -  Tg^(t)  +|Tg"(t)  +  ••■  +  (-1  ^  g^’^^  (t)  (64) 

where  r  is  the  order  of  the  polynomial  describing 
the  deterministic  signal 


Substituting  Eq  64  into  65  yields 


c(t) 


-  ^-jg'  (t)  +  •  •  • 

fT 

+  I  m(t  -  T)h( 

^0 


JrT 

n(t  - 

0 


(t) 

T)h(T)dT 


rT 

where  =  I  TT4i(T)dT,  v  =  0,  2,  •••  r 

Jo 


(65) 


Since  m(t)  and  n(t)  are  stationary  with  zero  means,  the  ensemble  average  output 
at  time  (t)  is  only  the  nonrandom  component,  g(t). 


c(t) 


HQg(t) 


T)h(T)dT 

Mig’Ct)  + 


+  (-I)""  5 


(66) 


If  the  desired  response  is  physically  realizable,  then  it  must  be  capable  of 
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being  related  to  the  total  signal  by  an  equation  of  the  form 


C(i(t) 


t)  +  m(t 


T)]hi(T)dT 


where  h^  =  an  "ideal”  weighting  function 
defined  by  Eq  6? 


(67) 


As  previously,  the  stationary  random  component  vanishes  when  an  ensemble  average 
is  taken 


Cd(t) 


T)hj^(T)dT 


For  the  error  to  have  zero  mean. 


(68) 


Cd(^) 


0 


(69) 


Substituting  from  Eq  66  and  68  into  Eq  69^ 


T)hj^(T)dT 


ij-og('t)  -  kiTg'('t)  +  •••  (-1)^  ^  g’(t) 


(70) 


Equation  70  determines  the  values  of  the  first  r  +  1  moments,  (1q,  |a>|  ,  •••  of 
the  optimum  impulsive  response,  h.*(T) .  In  Ref-  7  it  is  shown  that  the  ensemble 
average  of  the  time -averaged  square  error  (i.e.,  ^ /^Jq  e^dt)  can  be  expressed  as 


-  e)hi(T)hi(e)dTde 


J-ooJo 


T 

I^T  -  e)hi(T)h(e)dTd0 


- 


0)J  h(T)h(0)dTd0 


(71) 


where  and  are  the  autocorrelation  functions  of  m(t)  and  n(t) 
[There  is  assumed  to  be  zero  crosscorrelation  between  m(t)  and  n(t) | 


is  minimized  when  h(t)  satisfies  the  following  integral  equation: 
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rT 

I  +Riiin(^  -  T)]h(T)dT 

0 


The  procedure  for  detennining  Jq^  ,  •  •  •  7^  and  for  solving  Eq  71  is  described 
fully  in  Ref.  7- 

The  above  discussion^  and  that  of  the  previous  section,  has  been  concerned 
with  situations  where  the  command  is  partially  or  completely  deterministic. 

This  implies  well-defined  initial  conditions  (at  least  within  certain  limits) 
and  often  a  finite  time  of  observation.  These  conditions  apply  to  such  problems 
as  missile  interception,  orbital  rendezvous,  automatic  landing  control,  etc- 
For  these  problems,  terminal  errors  are  much  more  important  than  errors 
occurring  earlier.  Therefore,  it  seems  more  logical  to  employ  time-weighted 
criteria  (instead  of  minimum  e^)  so  that  the  analytical  optimization  procedure 
truly  reflects  how  the  importance  of  the  flight  control  system  fimction  varies 
with  time.  Such  time-weighted  criteria  are  discussed  in  the  chapter  that  now 
follows . 


=  7o  +  7i  t  +  •  •  •  +  /r't  + 


where  0  <  t  <  T 


Joo 
00’ 


Rm(t  -  T)hj^(T)dT 


(72) 


Jq)  7■^}  7r  are  Lagrange 

multipliers 
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CHAPTER  III 


TIME-WEIGHTED  CRITERIA 


Time-weighted  criteria  are  very  suitable  for  normalized  linear  constant - 
coefficient  systems  with  deterministic  inputs.  For  example^  criteria  such  as 


minimum  ITAE 


dt 


minimum 


it3e2 


min 


rco 

Jo 


tsp] 


2 

dt 


have  been  shown  (Ref.  1 )  to  yield  satisfactory  systems  for  step  inputs^  and  are 
highly  selective.  It  is  logical  to  inquire  whether  corresponding  criteria 
could  be  evolved  for  random  inputs.  This  chapter  investigates  this  problem  by 
examining  the  three  published  criteria  listed  below. 

1  .  Murphy  and  Bold^s  Criterion 


minimum  W(t)  [e(t)] ^dt 

where  W(t)  is  a  time -weighting  function  statistically 
independent  of  f[e(t)]  and  may  be  deterministic 


2.  Zaborszky  and  Diesel^ s  Probabilistic  Square  Error  Criterion 


minimum 


p(t) G^(t)dt 


where  e  (t)  is  the  ensemble  average  of  system  error 
squared  taken  over  a  number  of  trials^  and  p(t)  can 
be  interpreted  as  a  time-weighting  function  indi¬ 


cating  the  relative  in^ortance  of  errors  occurring 
at  different  times.  It  is  usually  permissible  to 
assume  the  trials  to  be  concurrent;  the  averaging 
sign  then  need  not  be  extended  over  p(t) . 
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5*  Glover *s  Criterion 


minimum 


T--00  2T 


€^(t) 

[=d(t)]"  +  6= 


dt 


where  is  the  desired  output^  and  6  is  a 

constant  which  dictates  the  lowest  absolute 
accuracy  of  interest 


The  principal  conclusions  of  this  chapter  are  that 

1  •  Murphy  and  Bold’s  criterion  yields  the  same  optimum  system  for 
all  choices  of  W(t)^  if  the  input  environment  is  stationary. 
This  is  true  even  when  the  criterion  is  generalized  by  replac¬ 
ing  ^  with  any  other  f(€) 

2.  the  probabilistic  square  error  criterion  is  suitable  for 
nonstationary  situations  (such  as  repeated  missile  trials) 
when  a  meaningful  selection  of  p(t)  can  be  made 

3.  Glover’s  criterion^  when  applied  to  free-order  linear  systems 
with  uncorrelated  Gaussian  input  and  noise ^  yields  optima 
closely  related  to  the  Wiener  system.  When  the  system  order 

is  constrained  (i.e.^  Phillips-type  optimization)^  the  criterion 
tends  to  be  unselective. 

These  conclusions  are  summarized  in  Fig.  21  . 

A.  MUKPHy  MD  BOLD’S  CRITERION 


In  Ref.  20^  Murphy  and  Bold  propose  the  MWSE  (mean  weighted  square  error) 
criterion 


minimum  MWSE 


minimum 


lim  J_ 
T^oo  2T 


W(t) G^(t)dt 


(75) 


W(t)  is  statistically  independent  of  €(t)^  and  may  be  deterministic.  This  crite¬ 
rion  is  a  special  case  of  the  more  general  criterion 


minimTim  MWFE 


minimum 


lim  _L  f  ^ 
T— cx)  2tJ_^ 


W(t)f  [e(t)]dt 


(7^^) 


62 


GENERAL  FORMS 


CRITERION 


SPECIAL 

CONDITIONS 


INPUT 

ENVIRONMENT 


CONCLUSIONS 


Figure  21  .  Summary  of  Tlme-Weignted  Performance  Measures 


Although  it  was  not  investigated  by  Murphy  and  Bold,  many  of  the  results  to  be 
proved  below  also  apply  to  this  form. 

Murphy  and  Bold  derive  the  expression  for  the  optimum  MWSE  system  by  a  pro¬ 
cedure  analogous  to  Wiener's.  A  sufficient  condition  for  the  MWSE  to  be  a  minimvun 
is  also  derived,  i.e., 

W(t)  >  0  for  -CD  <  t  <  oo 

which  one  would  intuitively  expect,  because  this  keeps  the  integrand  in  Eq  7^ 
positive.  Murphy  and  Bold  apparently  failed  to  realize  that  minimizing  the  MWSE 
is  equivalent  to  minimizing  (which  will  be  proved  below) .  This  point  also 
seems  to  have  been  overlooked  by  Zaborszky  and  Diesel  (Ref.  4-9)  in  their  comments 
on  Ref.  20,  and  in  the  accompanying  reply  of  Murphy  and  Bold.  The  equivalence  of 
minimizing  MWSE  and  is  proved  by  demonstrating  that 

MWSE  =  (€2)(w)  (75) 

Proof 

By  definition,  MWSE  =  W(t)e2(t)  (T6) 

vhere  the  bar  denotes  time  averaging 


Consider  W(t)  to  be  detemiinistic  with  a  finite  time-average  value  and  €(t)  to  have 
zero  mean.  Taking  the  ensemble  average  of  a  number  of  repeated  trials  yields  (using 
the  same  system  for  each  trial) 


E  [mWSe] 


(77) 


As  shown  on  p.  65  of  Ref.  50^  it  is  permissible  to  interchange  the  order  of  time 
and  ensemble  averaging  for  this  expression.  A  series  of  stationary  inputs,  each 
having  the  same  spectrum,  when  applied  to  identical  systems  yields  the  same  for 
each  system.  For  the  MWSE  criterion  to  be  of  practical  use,  it  is  likewise  neces¬ 
sary  that  when  a  series  of  stationary  inputs,  each  having  the  same  spectrum,  is 
applied  to  identical  systems  the  MWSE  values  for  each  system  must  be  eqml.  Thus 
e|mWSe1  =  MWSE  and  Eq  77  becomes 


MWSE  =  E[w(t)€2(t)] 


(78) 
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Since  W(t)  is  constant  when  averaging  across  the  ensemble  at  any  particular  value 
of  t,  then 

E|w(t)e^(t)J  =  W(t)E[e2(t)J  =  W(t)  a|  (79) 

p 

Because  (for  a  stationary  input  environment)  is  a  constant,  substituting 
Eq  79  into  Eq  78  yields 


MWSE  =  W(t)o|  =  o|(w) 


(80) 


Thus,  minimizing  MWSE  for  any  specified  deterministic  W(t)  is  singly  equivalent 

....  2 

to  minxmizing  a^. 

For  the  case  where  W(t)  is  a  random  signal  statistically  independent  of  e(t), 
the  proof  proceeds  as  indicated  above,  except  that  Eq  78  leads  directly  to 


MWSE  =  E(Wg2)  =  E(w)e(€2)  =  W(e^}  =  W  (e^)  (8l ) 

because  the  ensemble  average  of  the  product  of  two  statistically  independent 
variables  is  merely  the  product  of  their  individual  ensemble  averages. 

It  can  also  be  shown  that  minimizing  the  more  general  MWFE  criterion  is 
equivalent  to  minimizing  the  appropriate  f(€),  because  (by  a  similar  proof  to 
that  given  above) 

MWFE  =  DTiT](w)  (82) 

It  was  shown  in  Chapter  II  that,  for  many  input  distributions,  minimization  of 
the  very  broad  class  of  f (e)  measures  that  are  even  and  nondecreasing  with  \e\ 
is  equivalent  to  minimizing  Combining  this  result  with  Eq  82  leads  to  the 

observation  that  in  very  many  cases  the  MWFE  criterion  simply  yields  the  minimum 
G^  system. 
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B.  THE  "END-SIGMA"  AND  "PROBABILISTIC  SQUARE  ERROR"  CRITERIA 


Zaborszky  and  Diesel  first  proposed  the  end-sigma  performance  measure  in 
Ref.  22^  and  developed  it  further  in  Ref.  25^  24^  25^  26^  and  27.  This  measure 
is  defined  as 


(85) 


where  the  wavy  line  indicates  averaging  over 
an  ensemble  of  inputs  of  different  types 


The  integrand  contains  a  function  of  error,  time,  and  various  system  parameters 
^  ^2.^  ’’’  ^r^  P('t)^  the  probability  that  the  output  will  be  used  at 

time  t.  It  is  claimed  (Ref.  22)  that  "this  measure  xinites  in  a  single  concept 
the  transient  and  steady  states  of  the  system  operation  as  well  as  the  largely 
neglected  intermediate  states.  None  of  these  operating  states  is  discriminated 
for  or  against,  and  because  of  p(t),  each  gets  the  weight  due  it  because  of  the 
relative  frequency  of  its  presence  at  such  times  as  the  output  is  utilized."  The 
generality  of  the  measure  makes  it  impossible  to  dispute  this  claim,  and  credit  is 
certainly  due  to  Zaborszky  and  Diesel  for  recognizing  the  need  for  a  criterion 
which  is  compatible  with  both  deterministic  and  random  inputs.  However,  to  use 
the  criterion,  it  must  first  be  put  into  a  concrete  form.  The  particular  concrete 
forms  chosen  by  Zaborszky  and  Diesel  will  now  be  discussed,  and  validity,  selec¬ 
tivity,  and  ease  of  application  as  performance  criteria  will  be  assessed.  In  each 
of  Ref.  22,  25,  24,  25,  26,  and  27  the  measure  considered  was 


(84) 


where  p(t)  is  independent  of  €(t) 


which  Zaborszky  and  Diesel  call  the  "probabilistic  square  error."  This  form  is 
meaningful  only  for  nonstationary  situations  since  for  stationary  problems  the 
probabilistic  square  error  must  be  expressed  as 
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^QQ  -  jp  Kr\r\j\f\r\f\j 

S=  J^  P(t)[e(t)]^dt  =  Pi  (t)  [e(t)]^dt  (85) 

where  p(t)  =  ^[u(t)  -  u(t  -  T)jp^(t), 

and  Pi(t)  is  a  general  probability  distribution 
function 


If  p-|  (t)  is  independent  of  e(t)^  where  e(t)  is  a  random  function  of  time^  mini¬ 
mizing  ^  is  equivalent  to  minimizing  the  mean  square  error  (as  has  been  shown  in 
the  discussion  of  Murphy  and  Bold*s  criterion  in  Section  A  of  this  chapter. 

Thus  the  probabilistic  square  error  criterion  might  be  appropriate  for  non¬ 
stationary  situations  such  as  automatic  landing  systems  or  missile  flight  control 
systems  in  which  the  control  system  is  required  to  operate  for  limited  durations^ 
and  in  which  the  ensemble  average  performance  over  a  number  of  trials  constitutes 
the  basis  for  system  assessment. 

Evaluation  of  the  Probabilistic  Square  Error  in  the  s-Domain 


Two  procedures  for  evalixating  9  in  the  s -domain  will  now  be  given.  First 
the  procedure  suggested  by  Zaborszky  and  Diesel  in  Ref.  22  will  be  summarized^ 
and  then  an  alternative  procedure  which  is  often  more  convenient  for  simple 
systems  will  be  described. 

It  is  shown  in  Ref.  22  that  when  each  member  of  the  ensemble  is  the  same. 

=  [€(t)J  and  the  probabilistic  square  error  can  be  rephrased  as 

9=  f  p(t  -  t)[e(T)]^dT 

Jo 

=  t^o  ^  [p(-s)E(s)*E(s)] 


(86) 

(87) 


Numerator  and  denominator  polynomials  are  defined  such  that 


P(-s) 


A(s) 


(88) 
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and 


E(s)-E(s)  =  i[f} 


(89) 


and  it  is  demonstrated  that 


S  = 


i  [dL/ds] 

^“Pi 


(90) 


where  are  the  zeros  of  L(s)^  all  of  which 
are  assumed  to  he  simple  and  located  in  the 
left  half -plane 


Alternative  fomnulations  of  9  are  given  where  these  conditions  cannot  he  satis¬ 
fied  (Ref.  22). 

Example  of  Prohahilistic  Square  Error  Evaluation 


The  following  example  is  presented  hy  Zahorszky  and  Diesel  to  demonstrate  the 
computation  process  in  general:  a  system  described  hy  the  closed- loop  transfer 
function 


C(s) 

R(iy 


s  +  1 
+  S  +  1 


(91) 


is  subjected  to  an  input,  r(t),  where 

r(t)  =  1  +  2t 


(92) 


The  probability  function  of  the  output  is 


p(t) 


e-5^ 


) 


(93) 


This  example  was  solved  in  Ref.  22  using  the  procedure  described  above^  which  is 
quite  general  but  requires  the  zeros  of  L(s)  to  be  known.  However^  alternative 
methods  are  possible  for  an  example  of  such  simplicity.  (The  procedure  of  Mishkin 
and  Braun^  p.  of  Ref.  51^  is  worthy  of  mention;  but  this  also  requires 
knowledge  of  the  zeros  of  L(s).) 
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A  method  "will  now  he  demonstrated  which  has  the  advantage  of  obviating  the 
need  to  find  these  zeros.  Substituting  the  p(t)  of  Eq.  93  into  Eq  84  for  q 


S  = 


1_0 

3 


~dt 


(94) 


(95) 


Stone  (Ref.  52)  has  given  general  literal  forms  for 


E(s)j  . 


In  particular^ 


L 


C 


-1 


s  +  Bq 


s^  +  A.J  s  +  Aq 


B^s^  +  (B^A^  +  2BqB-|)s  +  (2B^Aq  +  2Bq) 

s^  +  5AjS^  +  (2A^  +  4Aq)s  +  4AqA^ 


(96) 


So^  in  the  present  example^ 


E(s)  = 


s  +  2 


s  +  2 


s"^  +  s  +  1 


s*^  +  s  +  1 


(97) 


Hence  the  probabilistic  square  error  is 

s=2 

^  10  2A  6o 
3  36  ‘  2^ 
s-5 

which  agrees  with  the  result  given  in  Ref.  22.  In  addition  to  the  fact  that  there 
is  no  need  to  evaluate  the  zeros  of  L(s)  using  Stone’s  formulas,  this  procedure 
may  also  be  more  convenient  for  minimization,  because  general  expressions  for 

,  S9/SBQ,  etc-,  can  be  derived.  However,  for  higher-order  systems  Stone’s 
formulas  become  exceedingly  long,  and  Zaborszky  and  Diesel’s  procedure  is  to  be 
preferred,  particularly  if  a  digital  computer  is  available  for  determining  the 
zeros  of  L(s). 


lof  +  3s  +  10  ~[ 

3  |_s2  +  5s^  +  6s  +  y 
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MiniTTii  zation  of  the  Probabilistic  Square  Error  in  the  Time  Domain 

In  Ref.  22  and  23^  Zaborszky  and  Diesel  discuss  the  evalimtion  and  minimiza¬ 
tion  of  probabilistic  square  error  using  time-domain  operations.  There  is  some 
correspondence  between  the  given  procedirre  and  Wiener  optimization,  but  the 
methods  adopted  by  Zaborszky  and  Diesel  to  solve  the  eqimtion  corresponding  to 
the  Wiener-Hopf  equations  are  essentially  approximate,  since  the  optimum  weight¬ 
ing  fimction  is  expressed  as 


n 


i=0 


where  n  is  predetermined 


The  accuracy  of  the  solution  for  the  optimum  weighting  function  is  thus  limited 
by  the  largest  value  of  n  that  can  be  handled  using  the  computational  aids  avail¬ 
able.  However,  time-domain  synthesis  is  advantageous  in  the  following  respects: 


1 .  nonexponential  forms  for  p(t)  can  be  handled  without  analytical 
difficulties.  (in  the  example  to  be  discussed,  p(t)  =  i/s  for 

1  <  t  <  6,  and  is  zero  outside  this  range.)  As  noted  on  p.  315 
of  Ref.  51,  such  a  form  would  be  analytically  inconvenient  for 
the  s- domain  procedures  discussed  above 

2.  the  physical  significance  of  each  step  in  the  optimization 
procedure  is  readily  apparent. 


For  convenience  of  digital  computer  prograinming ^  in  Ref.  22  the  inputs  desired 
output,  probability  weighting  function,  etc.,  are  approximated  by  sums  of  Legendre 
polynomials.  It  will  be  demonstrated  below  that  such  a  representation  is  not 
essential,  and  that  the  example  of  Ref.  22  can  be  worked  by  time-domain  procedures 
not  (explicitly)  involving  Legendre  polynomials.  This  simplification  enables  the 
physical  interpretation  of  the  time-domain  approach  to  be  kept  clearly  in  sight 
throughout  the  entire  optimization  procedure,  thus  fully  exploiting  the  advantages 
of  time-domain  synthesis.  The  error  can  be  defined  as 


(99) 


where  h(ti  )  is  the  system  weighting  function 
(k  is  used  rather  than  h  to  facilitate 
reference  to  Ref .  22) 
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With 


9  = 


r 

Jo 


CO  vAAAaaA/\AA/ 
p(t)  €^(t)dt 


(100) 


it  can  be  shown  (Ref.  23)  that 


'00 


=  1^0(10(1(0^  ^  J  ^(■tl)lfc^r(0^  -^)  + 


fOD  rco 

I  h(tj)(itj  I  1 

Jo  Jo 


h(ti  )\trpr(tj,  tj  -  t-|)dti 


where  tr^(t,  t) 


(101  ) 


rcD 

=  p(t2  +  t)r(t2)r(t2  +  t)  dt2 

Jq  p(^2  +  h)c^(t2)c^(t2  +  t)  dt2 
^00 

Jo  ^  h)c^(t2)r(t2  +  t)  dt2 


The  minimization  of  <;  is  effected  hy  equating  its  first  variation,  5q,  to  zero, 
which  leads  to 


oo 

h  (t-,  )ti.r('t^  t  -  ti  )dt^ 


-t) 


(102) 


The  resemblance  of  Eq102to  the  Wiener-Hopf  equation  should  be  noted.  However, 
in  the  Wiener-Hopf  equation,  the  term  corresponding  to  is  a  function  of 
t  -  t-j  only ,  so  the  techniques  that  have  been  evolved  for  solving  the  Wiener-Hopf 
eqxaation  cannot  be  applied  to  Eq  102. 

Zaborszky  and  Diesel  solve  Eq102by  approximating  k,  t„„,  and  with  s\jms 

iz  Cdr 

of  orthogonal  functions  such  as  Legendre  polynomials.  This  yields  n  simultaneous 
linear  equations,  where  n  is  the  order  of  the  highest  Legendre  polynomial  employed. 
In  general,  the  "optimum"  weighting  function  so  obtained  is  approximate,  in  that 
the  accuracy  of  the  approximation  is  limited  by  the  magnitude  of  n,  which  in  turn 
is  limited  by  the  capacity  of  the  computational  aids  available. 
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An  alternative  time -domain  procedure  for  obtaining  an  n'^^-order  polynomial 
approximation  to  the  optimum  weighting  function  will  now  he  outlined.  The  pro¬ 
cedure  is  most  easily  understood  hy  considering  an  example^  and  the  problem 
given  in  Ref.  25  will  be  used  for  this  purpose - 

The  example  discussed  in  Ref.  25  consists  of  the  optimization  of  a  system 
having  a  weighting  function,  ti(t) ,  such  that  the  output  will  be  a  prediction  of 
the  input  1  sec  from  the  present  time.  The  input  is 

r(t)  =  u(t)|^1  +  0.5t  +  (103) 

The  desired  output,  c^^,  is  thus  r(t  +  1 ) ,  which  becomes 

ca(t)  =  u(t)^3.5  +  4.5t  +  2t^  (104) 

The  probabilistic  square  error  criterion  is  used  with  equal  weight  attached  to 
all  outputs  occurring  between  t  =  1  and  t  =  6  sec,  and  zero  weighting  is  given 
to  outputs  occurring  outside  this  period,  i.e.,  p(t)  =  (1/5)  [u(t  -  1)  -  u(t  -  6)]. 
Hence,  the  quantity  to  be  minimized  becomes 


=  J  -  ■>)  -  u(t  -  6)Jjc(t)  -  c^(t)J' 


dt 


The  actual  output,  c(t),  is  given  by  the  convolution 


c(t)  =  r  h(T)r(t  -  T)dT 

Jo 


Following  Ref.  25,  h(t)  will  be  approximated  by  the  expression 


Ht) 


^0 


t 


+  k2t 


(105) 


(106) 


(107) 


Strictly  interpreted,  this  would  give  an  unstable  weighting  function,  but  because 
the  probability  that  the  output  will  be  used  is  zero  after  t  =  6  sec,  this  point 
is  of  academic  interest  only. 
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Performing  the  convolution^ 


A  necessary  condition  for 


9  to  be  a  minimuin  is  that 


which  must  be  satisfied  for  n  =  0  and  1  and  2 


This  is  simply  evaluated  by  noting  that  the  differentiation  can  be  brought 
the  integral  sign.  For  example^  with  n  =  0^  Eq  109  is  equivalent  to 


+ 


k- 


0 


This  gives  the  solution  of  =  0  in  terms  of  kQ,  k-j  ^  and  k2* 

Evaluating  this  integral  and  the  corresponding  equations  for 


Sk-j 


0. 


Sk2 


0 


(108) 


(109) 


under 


(110) 
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leads  to  a  set  of  three  simultaneous  equations  for  kQ,  k-j  ,  and  k2-  The  corre¬ 
sponding  eqiiations  derived  by  Zaborszky  and  Diesel  are^  in  matrix  fom^ 


r  4,392.06  -4,165.43  1,412.35" 

^0 

8,829.85" 

I  -15,206.65  13,815.24  -4,545.15 

^1 

= 

-29,021 .61 

L  3,558.83  3,188.88  i,355-96- 

-  k2  - 

.  7,912.02_ 

These  equations  are  mildly  ill-conditioned;  this  becomes  more  apparent  when  the 
coefficients  of  kQ  are  made  Tinity.  The  matrix  is  then 


1 

-0.964 

+0.3215 

'  ^0' 

■  2.01 

1 

-0.908 

+0.2987 

k1 

= 

1  .908 

_ 

1 

-0.901 

+0.3775  . 

-^2- 

2.236 

For  a  weighting  function  as  simple  as  that  considered  in  Eq105^  "the  ill- conditioning 
is  imlikely  to  be  troublesome^  but  it  will  probably  become  more  pronounced  as  the 
order  of  the  approximation  to  h(t)  is  increased.  However^  higher- order  weighting 
function  approximations  will  require  digital  computers  to  evaluate  the  simultaneous 
equations^  and  the  availability  of  digital  computers  will  ease  the  difficulties 
associated  with  ill- conditioning. 

The  time-domain  synthesis  procedure  presented  in  Ref.  23 is  essentially  directed 
towards  digital  computer  evaliiation,  and  differs  from  the  technique  described  above 
in  that  p(t),  ,  c^(t),  and  h(t)  are  represented  by  sets  of  orthogonal  func¬ 

tions.  The  resulting  expressions  are  fairly  complicated^  but  can  be  split  into 
two  parts^  one  of  which  is  invariant  with  p(t)^  ^(t)^  and  c^^(t)^  so  that  this  part 
can  be  retained  throughout  a  range  of  problems.  The  entire  programming  procedure 
is  tabulated  in  Ref.  25,  and  an  example  is  given.  An  extended  description  of  how 
a  homing  aircraft  flight  control  system  can  be  reduced  to  a  form  suitable  for 
^-optimization  is  given  in  Ref.  27-  In  Ref.  26^  a  continuous  control  system  is 
optimized  subject  to  a  constraint  of  acceleration  limiting  on  the  output,  c(t). 

The  resulting  nonlinearity  is  avoided  by  the  artifice  of  constraining  the  rms 
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output  of  a  linear  substitute  system  (actually  the  original  system  with  the  non¬ 
linearity  removed)  through  the  use  of  Lagrange  multipliers.  In  Ref.  2k  the 
probabilistic  square  error  of  a  sampled-data  system  is  considered^  and  extensive 
tabulations  are  given  of  the  required  digital  programming  procedure. 

Assessment  of  the  Probabilistic  Square  Error  Criterion 

It  is  hardly  possible  to  assess  such  a  generalized  perfomiance  measure  as 


without  making  a  number  of  arbitrary  choices  for  p(t).  In  the  present  generalized 
study^  no  sufficient  basis  exists  for  such  a  choice^  but  it  is  at  least  arguable 
that  in  specific  applications  this  choice  will  be  easily  made.  Accepting  this 
premise  (together  with  the  consequence  already  mentioned  that  for  some  p(t)^ 
unstable  systems  may  result  from  the  optimization  procedure)^  it  appears  that  the 
probabilistic  square  error  criterion  and  its  generalized  form,  the  end- sigma 
criterion,  are  quite  suitable  for  the  optimization  of  nonstationary  systems.  The 
present  report  has  attempted  to  indicate  ways  in  which  the  criterion  may  be 
handled  using  desk  rather  than  digital  computers.  In  particular,  a  suggested 
procedure  for  s-domain  synthesis  of  low-order  systems  has  been  discussed.  For 
the  analysis  of  flight  control  problems,  it  is  usually  permissible  to  replace  the 
actual  flight  control  system  by  a  low-order  equivalent  system  (page  7),  and  the 
use  of  this  substitution  should  render  Zaborszky  and  Diesel* s  criterion  suitable 
for  "pencil  and  paper"  studies. 

C.  GLOVER  *S  AiyiPLITUDE-WEIGHTED  PERFORMANCE  MEASURE 

In  Ref.  21  Glover  reasoned  that  "the  criterion  of  least  mean  square  error 
has  the  disadvantage  that  large  effects  are  weighted  quite  heavily  even  when 
they  occur  at  a  time  when  the  variable  under  consideration  is  large."  He  further 
reasoned  that  usually  the  error  expressed  as  a  percentage  of  the  desired  output 
is  of  more  interest  than  is  the  absolute  error,  and  therefore  proposed  a  perform¬ 
ance  measure  in  which  the  error  time-weighting  fiinction  contains  the  amplitude 
of  the  desired  output.  For  convenience,  the  performance  measure  will  be  referred 
to  as  P.E.  because  the  integrand  involves  error  expressed  as  a  percentage  (or 
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fraction)  of  the  desired  output: 


P.E- 


/ 


(115) 


where  €(t)  is  the  error 


c^(t)  is  the  desired  output 


6  is  a  constant  which  dictates  the 
lowest  absolute  accuracy  of  interest. 
It  prevents  the  integrand  from  becom¬ 
ing  infinite  whenever  c^^(t)  =0. 


Glover  used  the  criterion  of  minimum  P.E.  with  the  restriction  that  only 
linear  time -invariant  systems  would  be  considered.  He  showed  that  with  uncorre¬ 
lated  Gaussian  signal  and  noise ^  the  optiminn  linear  system  is  merely  the  Wiener 
system  multiplied  by  a  constant.  Glover’s  analysis  is  outlined  below^  and  an 
extension  is  then  derived  for  fixed-order  (Phillips-type)  optimization.  The 
principal  conclusions  of  this  section  are : 


1  -  because  the  Glover  optimum  linear  system  is  so  simply 
related  to  the  Wiener  system,  it  suffers  from  the 
possible  disadvantages  of  the  latter,  e.g.,  poor 
transient  response 

2.  for  Phillips-type  optimization,  Glover’s  performance  measure 
tends  to  be  even  less  selective  than  the  minimum  mean  square 
error  criterion. 


General  Expression  for  P.E.  and  the  Optimum  Linear  System 

Glover’s  procedure  for  the  optimum  linear  system  will  be  outlined  briefly 
below,  while  simultaneously  manipulating  P.E.  into  a  form  suitable  for  Phillips- 
type  optimization. 

The  expression  for  P.E.  (Eq113)  can  be  written 
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P.E. 


(11^) 


2Cjj^c  +  c 


+ 


2 


where 

c 

h(x) 


f(t  +  Tl) 

fj"  h(x)r(t  -  x)dx 

the  impulsive  response  of  a  general 
linear  physically  realizable  system 


Now  let  W(t)  = 


“2  "be  the  weighting  function  on  (Eq  llj)  - 


[f(t  +  t])]2  +  6^ 
tuting  into  Eq  the  expressions  for  c,  and  W  yield 


P.E. 


x)f(t  +  Ti)W(t)  dx 


+ 


h(x)h(y)  r(t  -  x)r(t  -  y)W(t)  dxdy 


Glover's  definition  of  a  "weighted  correlation  fimction," 


%uv('^l  )  "^2^  ^  u(t  +  T-|)v(t  +  T2)W(t) 


allows  P.E.  (Eqll^  to  be  expressed  compactly  as 


P.E. 


0)  -  2 


li(x)%rf[-(x  +  Tl),  0]dx 


^00  ^00 

)  h(x)h(y)l^  [-(x  +  Tl),  -(y  +  Ti)]dxdy 

•'0  •'0 


Suhsti- 


(115) 


(116) 


Cut) 
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Variational  calculus  methods^  as  in  the  Wiener  derivation^  are  used  by  Glover 
to  establish  that  for  P.E.  to  be  a  minimum^  it  is  necessary  that  h(x)  satisfy  the 
integral  equation 


J^oo 

h(y)R;jrrt'(^  +  tj),  -(y 

n 


Ti)]dy 


(118) 


Reference  21  indicates  that  the  solution  of  the  integral  equation  would  be 
immensely  simplified  if  the  weighted  correlation  function  could  be  expressed  in 
terms  of  the  ordinary  correlation  functions.  This  will  now  be  accomplished  using 
the  ergodic  theorem  to  express  ^  T2)  in  terms  of  ensemble  averages. 

Determination  of  the  Optimum  Weighting  Function 

for  Gaussian  Input  and  I^oise 


Glover  expresses  the  weighted  correlation  function  as  ensemble  averages  by 
the  relation 


■^2) 


-00  +  o 


c^) dudvdc^ 


(119) 


where  p(uj  v^  c^)  is  the  joint  probability 
density  function^  and  u  and  v  are  the 
general  variables  used  in  Eq  1 1 6 


Substituting  this  form  of  weighted  crosscorrelation  function  into  Eqll8  leads  to 
solutions  for  the  optimum  weighting  function  when  the  distributions  involved  can 
be  specified  analytically.  For  Gaussian  input  environments,  there  are  three 
cases  of  interest; 

zero  noise 

uncorrelated  signal  plus  noise 
correlated  signal  and  noise 


1  . 
2. 
3. 
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Zero  Noise. 


For  this  case^  Glover  obtains 


*^2)  “  Pff  ('^1  )  Pff  (‘^2)  +  1^2^ff('^l  ”  ^2) 


(120) 


where  ki 


pff(T)  = 


,  .JI(±^^\  erfe  -2- 

V2(,0f  sj"  0fV^ 


■  m) 


.52/2af 


erfc 


afi 


normalized  autocorrelation  function 
of  the  signal  (as  in  Eq  125) 


Glover  does  not  present  the  rather  lengthy  manipulations  required  to  obtain 
Eq  120;  however,  it  has  been  verified  independently  by  the  present  authors. 

The  optimum  system  found  from  Eq  1 1  8  and  1 20  is  the  ordinary  Wiener  system 
multiplied  by  a  constant,  i.e.. 


(121) 


where  ^opt^^^ 


k  = 


B  = 


A  = 


optimiom  weighting  function 
B  +  (1  -  B)jr”  lv(y)Pff(y  +  Ti)dy 


Of 


yl  erfc 


Of 


=  variance  of  signal 


h^(t)  =  Wiener  optimum  predictor  weighting 

function 

Pff(y  +  T])  =  normalized  autocorrelation  function 

erfc  X  =  the  complementary  error  function 
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Signal  Plus  Uncorrelated  Noise.  The  general  form  of  the  solution  for  n(t)  =0 
also  applies  to  the  case  where  n(t)  0^  providing  that  n(t)  =  0^  and  that  there 

is  no  crosscorrelation  between  signal  and  noise.  The  optimum  weighting  function 
is  again  of  the  form  of  Eq  121  where  now  hy(t)  is  the  Wiener  weighting  function  for 
the  given  signal  and  noise. 

Correlated  Signal  and  Noise.  Glover  states  that  attempts  to  obtain  similar 
solutions  both  when  the  noise  and  true  signal  are  correlated^  and  when  using  other 
types  of  statistics^  lead  to  the  necessity  of  solving  higher-order  transcendental 
equations,  which  can  probably  only  be  solved  by  recourse  to  machine  computations. 

It  is  a  striking  fact  that  for  Gaussian  inputs  and  noise,  the  optimum  system 
is  simply  the  Wiener  system  multiplied  by  a  constant  (Eq  I2l)  .  Thus,  if  the  optiminn 
Wiener  system  for  a  given  Gaussian  environment  is  unsatisfactory  due  to  poor 
transient  characteristics,  replacing  the  criterion  by  Glover *s  criterion  will 
effect  no  significant  improvement.  The  corresponding  relationship,  if  any,  for 
non-Gaussian  environments  has  not  yet  been  determined  due  to  the  great  difficulties 
associated  with  general  analytic  evaluations  of  Eq  117 

Glover^ s  Criterion  Applied  to  Fixed-Order 
Systems  (Phillips-Type  Optimization)" 

General  expressions  for  the  optimum  P.E.  system  of  fixed-orxier  (Phillips 
system)  will  now  be  derived.  (This  problem  was  not  considered  by  Glover.)  A  typi¬ 
cal  example  is  evaluated,  and  the  P.E.  performance  measure  is  shown  to  be  even  less 
selective  than  the  minimum  mean  square  error  criterion. 

Zero  Noise.  Using  the  egression  for  P-'^ff('i'i  ^  ^^2)  gi'V'en  by  Eql20,  and  insert¬ 
ing  it  into  Eq  115^  gives  the  P.E.  as 


jk-|  Pff.(0)Pff(0)  +  k2Pff(0)j  -  2  J  h(x)  |pff(-x)Pff(0)k-|  +  P^j(-x)k2jdx 

^00  ^00  r 

+  J  J  h(x)h(y)  |Pf|.(-x)pff(-y)k^  +  Pff.(y  -  x)k2l<iydx;  (l22) 
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The  integrals  are  merely  convolutions  evaluated  at  zero^  the  second 

term  is 


lim 

t-^0 


h(x)p^l*(t  “  x)dx 


(125) 


Using  the  following  properties  of  normalized  autocorrelation  functions^ 


The  P.E.  becomes 


Pff(O) 


P„(t) 


1 


~ 


(124) 

(125) 


P.E. 


(k-|  +  k2)R^^(0) 
2 


2(kT  +  k2)Rjp^(0)  k^Rfc(O) 

2  4  2 

CT^ 


(126) 


This  can  be  rewritten  by  adding  and  subtracting 


kiRcc(O) 

2  • 

Of 


P.E. 


(ki  +  k2)[Rff(0)  -  2Rfc(0)  +  Rcc(O)]  + 


2 

^f 


[Rfc(o)]^ 

- 2 - ^cc(O) 


(127) 


.  p  -o  rR?n(o)  1 

..  0^  P.E.  =  (k^  +  k2)e2  +  -  Rcc(0)J  (128) 

where  is  the  mean  square  error  for 
this  (zero  noise)  case 


It  can  be  seen  from  Eq  128  that  in  general  the  optimum  system  will  not  be  a 
constant  times  the  minimum  system  (unlike  the  case  when  the  system  order  and 
form  were  left  free  during  the  minimization) .  This  can  be  seen  from  setting 
SP,E*/Baj_  =  0  where  aj_  is  a  variable  parameter  of  the  system* 
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Signal  Plus  Uncorrelated  Noise.  Equation  128  for  P.E.  1)60011105^  in  this  case. 


2 


P.E. 


+ 


Rfe(O) 

of 


(129) 


vhere  is  the  mean  square  error  for  the 

particular  uncorrelated  signal  and 
noise  present 

1  P 

J-co 


Again  it  can  he  seen  that  in  general,  minimizing  P.E.  vill  not  lead  to  the  same 
parameters  of  H(j(D)  as  minimizing  6^. 

Signal  Plus  Correlated  Noise.  The  expression  for  P.E.  for  this  case  will  he 
similar  to  those  of  the  previous  cases.  The  result  will  not  he  detailed  here 
because  the  expressions  are  very  lengthy.  However,  the  form  of  the  P.E.  expres¬ 
sions  has  heen  studied,  and,  as  in  the  case  of  signal  plus  uncorrelated  noise, 
it  is  such  that  the  optimum  P.E.  system  is  in  general  not  simply  related  to  the 
optimum  system. 

Example  of  Phillips-Type  Optimization  Using  Glover *s  Criterion 

As  an  example  of  Phillips-type  optimization,  consider  a  second-order  system. 


H(s) 


1 

s^  +  2^s  +  1 


with  a  signal  input  spectrum 


^nn(“)  =  0 


(130) 


(151) 


The  variation  of  Glover ^s  performance  measure  for  this  system  is  illustrated  in 
Fig.  22.  In  Fig.  23,  Glover *s  criterion  is  directly  compared  with  the  criterion 
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p 

of  minimum  e^.  This  comparison  has  "been  effected  by  combining  the  representa¬ 
tive  b/a^  =  1  .0  graph  of  Fig.  22  (which  corresponds  to  6  =  rms  input  amplitude) 
with  the  a  =  0.25  graph  of  Fig.  5^  which  corresponds  to  an  input  cutoff 
frequency  of  O.25  of  the  system  undamped  natural  frequency.  Figure  23  shows 
that  Glover's  criterion  is  less  selective  than  the  minimum  criterion.  But 
the  selectivity  of  the  criterion  is  only  barely  acceptable.  Therefore, 
Glover's  criterion — due  to  its  poor  selectivity — must  be  regarded  as  unsatis¬ 
factory. 
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Input  spectrum  <J>rr(cj)  = 


Input  R.M.S.  value  =  cTr,  where 


Unit  numerator  second -order  system  H(s)  = 


s^  +  25s+  I 

Note:  Criterion  is  multiplied  by  •§:  to  give  compact  plot 


X  Glover's  Criterion:  Carpet 


im  J_ 

2T  J  ^ 


g^(t) 


[cd('t)]  +  5^ 


dt  vs  ^  and 


P.E.  and 


- 9;  a  =  0.25;  S=  1.414  ;  o-.  =  1.414;  b  =  l 

r  I  Q  ^  * 

H(s)  =  -  '  ' - 

s2  +  2Cs  +  I 


0.5 


Figure  23.  Comparison  of  Glover *s  (P.E.)  Criterion  and  Mean  Square  Error 
of  Second-Order  Unit  Nuimerator  System 


CHAPTER  IV 


THE  GENERALIZED  ERROR  FUNCTION 

This  chapter  is  primarily  concerned  with  the  generalized  error  function 
(G.E.F.)  which^  for  stationary  random  inputs  (zero  noise) ^  is  defined  as 

/*+T  /"H-T 

[r(t  -  T)  -  c(t)]2dt 

(152) 

The  G.E.F.  has  a  compatible  form  for  deterministic  inputs,  which  Ref.  Jl  denotes 
by  E^: 

rco  rco 

Et  =  /  [e(t,  t)]  dt  H  /  [r(t  -  t)  -  c(t)]^dt  (155) 

Jo  Jo 

As  defined  in  the  Introduction^  compatibility  implies  that  E^  and  G.E.F.  will  be 

numerically  equal  for  any  given  linear  constant- coefficient  system,,  provided  that 

the  appropriate  deterministic  input  used  in  computing  E-^^  is  the  transient  analog 

of  the  random  input.  It  has  already  been  shown  (p.  l4)  that  [g.E.F.^  and 

Q  are  compatible  for  the  case  of  a  imit  numerator  second-order  system  when 

the  random  input  has  the  power  spectrum  =  1 /(ac^  +  a^)  and  the  deterministic 

—at 

input  is  e  .In  this  chapter  it  will  be  shown  that  the  G.E.F.  and  are  com¬ 
patible  for  all  linear  constant- coefficient  systems^  i.e.^  the  transient  analog 
of  a  random  input  does  not  depend  upon  the  system  to  which  it  is  applied.  This 
is  demonstrated  in  Section  A  below  by  expansion  of  the  analysis  of  Benedict  and 
Rideout  (Ref.  29),,  who  considered  only  the  case  of  zero  noise.  Section  B  derives 
general  transient  analogs  for  both  uncorrelated  and  correlated  signal  and  noise 
inputs.  These  results  are  used  extensively  in  the  third  part  of  the  chapter  to 
calculate  and  form  an  assessment  of  the  G.E.F.  Since^  for  t  =  0^  G.E.F.  = 
the  transient  analog  concepts  apply  equally  well  to  the  optimization  of  systems 
using  the  mean  square  error  criterion. 

A.  TRANSIENT  ANALOGS  (ZERO  NOISE) 

For  the  G.E.F.  and  Et  performance  measures  to  be  compatible^  the  transient 
analog  input  must  have  an  energy  spectral  density  that  is  the  same  function  of 
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frequency  (except  for  the  units)  as  the  power  spectral  density  of  the  random 
input  signal.  This  relationship  was  first  noted  hy  Schultz  and  Rideout  (Ref.  28). 
A  proof  is  given  below. 

The  Generalized  Error  Function  for  either  random  or  deterministic  inputs  can 
be  thought  of  as  the  mean  square  error  where  the  error  is  defined  in  Fig.  2^. 


For  a  transient  determiinistic  inputs  r^^  commencing  at  t  =  0,  (Fq  135)  can 
be  expressed  as 


Et(-^) 


T)c-j.(t)dt 


2 


dt 


(154) 


The  corresponding  expression  for  the  G-E.F.  (Eq132)  for  a  stationary  random 
inputs  r^  with  output  c^  can  be  written  in  terms  of  auto-  and  crosscorrelation 
functions  as 

G.E.F.  =  R^r(O)  -  2R^c(t)  +  R^^(O)  (l  55) 


It  will  now  be  shown  that  Eq  1  5^  and  1  35  sire  identical  term  by  term,  if  the 
deterministic  input,  r-(-(t),  is  the  transient  analog  of  the  random  signal,  r(t) 
(the  energy  spectral  density  of  r^(t)  being  equal  to  the  power  spectral  density 
of  r(t)).  Taking  the  last  term  first,  by  Parseval’s  theorem. 


^  I  Ct(s)Ct(-s)ds  =  ^  I  H(s)R^(s)H(-s)Rt(-s)ds 
J-joo  J-jco  ,  , 


(156) 


where  H(s)  =  the  system  transfer  function  in  Fig.  24 

Rt(s)  =  /^[i-t(t)] 
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Substituting  s  =  jcD^  Eq  1  56  can  be  -written  in  terms  of  the  input  energy  spectral 
density  as 


2  1  O 

Jo  •/-oo  ^  ^ 


(157) 


where  ^  (cd)  =  I^(  ja))R-j-(- jcd)  ,  the  transient 

input  energy  spectral  density 
(p.  102  of  Ref.  50) 


^t^t 


The  last  term  of  the  G.E.F.  (Eq  135)  can  be  written  as 


roo  1  r°° 

I^cc(O)  =  ^  I  ^  I  H(jcD)H(-ja))<I>j.3^(a))da)  (l 

J-00  J~co 


38) 


and  if 

<J>  (cn) 
rr 

= 

f  (“) 

I-t^t 

(139) 

then  from  Eq  1  37  1  38  it  can  be 

seen 

that 

/ 

*00  „ 

Rcc(O) 

=  / 

[ct(t)]  dt 

(i40) 

^0 

Similarly, 

the  first  terms  of  Eq  1  3i^-  and  135  are  equal. 

roo  p 

I  [^tC't  -  "r)]  = 

Jo 

2itj  _ 

r*  [e'^%(-s)]ds 

/-joo 

(l4i) 

2jtj 

r  joo 

= 

R^(s)Rt(^s)ds 

(i42) 

/-joo 

•00 

~~ 

(143) 

and,  if  the 

energy  spectrum  of  r^(t)  is 

the  same  function  of  03  as  the 

pover 

spectrum  of  r(t), 
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J_ 

2jt 


<J>^^(oD)dCD 


{^kk) 


The  middle  term  of  Eq  1  5^  can  be  shown  to  be  equal  to  the  middle  term  of  Eq  1  35 
by  using  the  following  version  of  Parseval’s  theorem  (Ref.  6^  p.  -4-3): 


rco 

Jo  ' 


x-i  (t)x2(t)dt  = 


1 

2^3  I  XT(-s)X2(s)ds 


(145) 


-joo 


where  X-]  ( s )  =  [x-|  ( t )] 

X2(s)  =y'[x2(t)] 


Substituting  r-j-(t  -  t)  for  x-|  (t)  and  c-j-(t)  for  X2(t)  yields  the  middle  term  of 
Eq  1 34  as 


rco  rjco 

I  r^(t  -  T)c^(t)dt  =  ^  I  e  R^(-s)Ct(s)ds 

Jo  J-joo 


where  X-j  (s)  =  R{.(s)e 
X2(s)  =  Ct(s) 


-TS 


(146) 


Equation  1 4-6  can  be  written  as 


X 


00  rjoo 

r^(t  -  T)ct(t)dt  =  I  H(s)Rt(s)R^(-s)e'^®ds 

^  J-joo 


(147) 


Substituting  s  =  jcd, 


Joo 

j 


=  ^  I  H(ja.)^^^^^(<D)eJ  dcD 


(148) 


Row  the  middle  term  of  Eq  1  35  is  (by  definition) 


Rrc('^) 


-  f” 

J-co 


and  since  <&^^(ju))  =  H(  jcD)'3>j,j,(cD) ,  Eq  l49  becomes 


(i49) 
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(150) 


^rc ( ^ ) 


■if" 

•7-00 


H  ( j  co)  ( oj)  ^dcD 


If  the  energy  spectrum  of  r^(t)  is  the  same  function  of  frequency  as  the  power 
spectrum  of  r(t) ,  then  Eq  150  and  l48  are  equal,  i.e.. 


t) c^(t)dt 


RrcC-r) 


(T51  ) 


Equations  13^  and  135  are  then  identical  term  by  term  for  all  H(s)  if  r^(t)  is 
the  transient  analog  of  r(t),  i.e.,  if<I>j,^(aD)  =  S  (m) . 

A  transient  signal  whose  energy  spectral  density  is  the  same  fimction  of 
frequency  as  the  power  spectral  density  of  any  stationary  random  signal  can  he 
obtained  as  indicated  by  Fig.  25  where  the  white  noise  generator  has  unit  power 
per  cps. 


Figure  25.  Generation  of  Transient  Analogy 
The  random  signal  spectrum  is 


<j5rr(«^)  =  Hi  (-jod) 

while  the  transient  signal  is 

r^(t)  =  h^ (t) 

Ht(s)  =  Ht(s) 

and  therefore  its  energy  spectral  density  is 

'^P^r^(^)  ~  j^)  =  H-j  ( jCD)Hi  ( - jcu) 


(152) 


(153) 


(154) 
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Before  proceeding  to  describe  transient  analogs  for  signal  plus  noise  inputs 
(correlated  or  uncorrelated) ^  tvo  minor  points  must  be  noted.  Throughout  the 
above  analysis  it  has  been  tacitly  assumed  that  the  value  of  t  is  fixed.  In 
fact^  the  selection  of  t  is  not  a  simple  matter.  The  reader  may  also  have 
observed  that  for  an  integrated  white  noise  input  (which  has  a  step  as  its 
transient  analog) ^  Eq  1  34  cannot  be  evaluated  directly  because  the  integrals  on 
the  right  fail  to  converge.  This  obstacle  can  be  easily  circumvented.  Discus¬ 
sion  of  both  these  details  is  delayed  until  later  in  this  chapter  to  avoid  too 
great  a  digression  from  the  present  topic  of  transient  analogs  for  signal  and  noise. 

B.  TRANSIEOT  MALOGS  FOR  UTJCORRELATED  SIGNAL  AND  NOISE 

The  G.E.F.  for  a  system  subjected  to  uncorrelated  signal  and  noise  can  be 
obtained  very  simply  by  applying  transient  analogs  of  the  signal  and  of  the  noise  at 
widely  separated  times^  as  will  now  be  shown. 


Figure  26.  Block  Diagram  for  Uncorrelated  Signal  and  Noise  Inputs 


For  the  system  illustrated  in  Fig.  26^  the  error  power  spectrum  is  (from  p.  50) 

=  h  -  H(jm)  |^43gg(cD)  +  |h(  jco)  ]  (155) 


Zero  prediction  or  lag  (t  =  O)  will  be  assimied  to  keep  the  analysis  brief ^  although 
there  is  no  essential  difficulty  in  extending  it  to  cover  the  case  of  t  ^  0. 

With  T  =  0^ 


G.E.F. 


(156) 


The  transient  analog  of  the  input  will  now  be  shown  to  consist  of  the  transient 
analog  of  the  signal,  s-t(t),  plus  that  of  the  noise,  n-t(t),  where  n^(t)  is  applied 
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to  the  system  only  after  the  error  transients  produced  hy  s-t(t)  have  become  effec¬ 
tively  zero.  Therefore,  the  lE^  for  these  transient  signals  is 


IE' 


.2  _ 


fco 

Jo 


[£T(t)  +  €2(t  -  T)]^dt 


(157) 


where  e-j  (t) 
e2(t  -  T) 
T 


is  the  error  due  to  the 
inputs  s^(t)  applied  at 

is  the  error  due  to  the 
inputs  applied  at 

is  much  longer  than  the 


signal  transient  analog 
t  =  0 

noise  transient  analog 
t  =  T 

system  settling  time 


Since  €2(1  -  T)  only  occurs  long  after  the  initiation  of  G'j(t)^  there  will  be  no 
c  ro  s  sproduc t ;  the  ref ore  ^ 


TT  p  roo 

Ie2  =  I  [€^(t)]"^dt  +  I  [e2(t  -  T)]^dt  (158) 

The  first  and  second  integrals  are  approximately  lE^  and  lE^;  therefore^ 

r(X)  ^00 

Ie2  1  j  e^dt  +  J  e|dt  (159) 

and  using  Parseval's  theorem,  this  can  be  written  as  (defining  Ei(s)  (t)  I ) 

o  1  1 

IE  =  I  E-I  (s)E-]  (-s)ds  +  2^  I  E2(s)E2(-s)ds  (l60) 

J-joo  J-joo 

E^ (s)  and  E2(s)  can  be  found  from  the  definitions  below  Eg  157,  and  Fig.  26. 
Ie-jCs)!  and  |E2(s)1^  can  then  be  expressed  in  terms  of  the  signal  or  noise 
energy  spectral  densities. 

^(s)S(-s)  where  B(s)  =  jC  ft(i')] 

^  (i6i) 


S(s)S(-s) 

where 

S(s) 

S  (“)  = 

N(s)N(-s) 

where 

N(s) 

Equation  1 60  becomes 

f  joor 

I  [h  -  Hi 

“joo*- 

2  e 

Os^st 

+ 

(162) 
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Since  ^gg  =  ^g^g^  and  ^>nn  =  ^n-tn-^^  integrand  in  Eq  1  62  equals  the  right  side 

of  Eq  155  term  by  term^  and  therefore  lE^  = 

Example  of  Transient  Analogs  for  Uncorrelated  Signal  and  Noise.  A  random 
signal  having  the  power  spectrum  <J>ss(a))  =  ^/{a^  +  a?)  has  a  transient  analog 
St('^)  =  [because  the  energy  spectral  density  of  this  transient  is 

I S^(s)S-^(-s)  I  =  l/(a^  +  Thus^  the  produced  by  a  stationary  random 

signal,  ^  presence  of  white  noise,  $^^(0))  =  1,  can  be 

obtained  by  applying  the  inputs,  st(t)  =  and  a  delayed  unit  impulse, 

^t(^)  “  ""  T)  ^  integrating  the  error  squared  of  the  transient  response. 

The  shaded  area  of  Fig.  27  indicates  the  parts  of  the  error  response  that  con¬ 
tribute  to  the  total  lE^. 


Figure  27*  Transient  Analogs  for  Signal  and  Noise 


C.  TRANSIENT  ANALOGS  FOR  CORRELATED  SIGNAL  AND  NOISE 

The  transient  analog  for  correlated  signal  and  noise  is  the  transient  analog 
for  uncorrelated  signal  and  noise  plus  an  additional  transient  input  to  a  modified 
system  that  simulates  the  effect  of  the  correlation.  This  modified  system  can  be 
varied  independently  of  the  signal  and  noise  spectra  to  determine  the  effect  of 
the  crosscorrelation  on  the  G.E.F. 

For  the  system  of  Fig.  26  with  correlated  signal  and  noise  (Ref.  50,  p.  239), 
the  error  power  spectral  density  is 

=  D  -  -  (1  -  -  (1  -  H)hXs  (165) 
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where  the  following  abbreviated  notation  is  used  to  keep  the  expressions  reason 
ably  concise: 


iHr  = 

H  = 
r(jCD)  = 

and  the  mean  square  error 


HH 

^sn 

H(j<x>) 

^ns 

H(-jcn) 

^ss 

is 

^nn 

e2  -  -L 

~  2n  ^ 

roo 

"-00 

=  <J'ns(j“)  (l64) 

=  ^ss(“) 

(165) 


The  first  two  terms  of  Eq  1  63  do  not  involve  the  crosscorrelation,  and  their 
transient  analogs  can  be  obtained  by  the  procedure  that  is  described  in  Section  B. 
The  contribution  of  the  last  two  terms  of  Eq  1  63  to  can  be  expressed  as 
follows : 


1  r  ®  foo 

^  ^  I  (1  -  (166) 

J-co  J-co 

It  is  assumed  that  =<& is  a  rational  function  of  joo,  i.e., 

=  Ha(jcD)H^(-ja))  =  (167) 

where  Hg^  is  a  system  transfer  function  composed  of  the 
left-half- s-plane  poles  and  zeros  of 

is  similarly  composed  of  the  right-half- s-plane 
poles  and  zeros  of 


Inserting  Eq  1  67  into  Eq  1 66  and  substituting  s  =  jcD  yields  (after  rearranging 
terms) 

^  I  .  -H*)H^,H^Hds  -  ^  -  H)ds  (l68) 

^-JOO  ^  J-joo 

Using  a  version  of  Parseval's  theorem  (Ref.  6,  p.  43), 
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(169) 


fco  rjoo 

J  X-]  (t)x2(t)dt  =  (-s)X2(s)ds 

where  X-j  (s)  =  /I  [x-,  (t)] 

X2(s)  =x:[x2(t)] 

it  can  he  shown  that  is  twice  either  term  in  Eq  1  68  : 

Ae^  =  _2 

This  can  be  seen  from  Eq  1  68  and  1  69  .  In  Eq  1  68  the  integrands  are  complex 
conjugates.  Identifying  the  asterisked  quantities  (l  -  H^)  and  in  the  first 
integrand  of  Eq  1  68  with  X-j  (-s)  and  the  remaining  quantities  with  X2(s)  reveals 
that  the  first  integral  of  Eq  1  68  is  equal  to  the  left  side  of  Eq  169.  A  similar 
identification  can  be  made  for  the  second  integral  of  Eq  168.  Therefore^ 
these  integrals  are  equal.  Equation  170  for  is  now  in  a  form  suitable  for 
computation  by  means  of  operations  upon  transient  quantities.  A  circuit  suit¬ 
able  for  the  synthesis  of.  is  illustrated  in  Fig-  28 y  where  transient  inputs 
s)  and  are  generated  simultaneously  and  fed  through  systems 

having  transfer  functions  H(s)  and  1  -  H(s) .  The  resulting  signals^  and 

€-j3(t)^  are  multiplied  and  integrated  to  give  This  can  be  seen  by  inspec¬ 

tion  of  Fig.  28  and  use  of  Eq  1  69  to  yield  Eq  170.  Note  that  since  Hg^(s)  and 
Hb(s)  both  have  left-half -plane  poles  only^  there  is  no  problem  with  regard  to 
stability  or  physical  realizability  of  this  circuit. 


1 

2nj 


f 


JOO 


(1  -  H^)H^HaHds 


JOO 


(170) 


Figure  28. 


Circuit  for  Generating  Contribution  to 
Due  to  Crosscorrelation  of  Signal  and  Noise 
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Etkin  (Eq  2.17  throTigh  2.19  of  Ref.  44)  derives  an  incorrect  general  form 
for  the  transient  analog  of  a  system  subjected  to  two  concurrent  stationary 
random  inputs  having  power  spectra  (cd)  2(^)9  ^  cross  spectrum^  ^  2(J^)  • 

(These  could^  of  course^  represent  correlated  signal  and  noise.)  His  analysis 
is  valid  only  when 


(171) 


There  is  no  reason  why  Eq  1 7”!  (which  corresponds  to  Eq  2.19  of  Ref.  44)  should 
be  satisfied  in  general^  since  it  says  that  the  cross-spectrum  between  signal 
and  noise  always  has  the  same  left-half -plane  poles  and  zeros  as  'the  noise 
spectrum  and  the  same  right-half -plane  poles  and  zeros  as  the  signal  spectrum. 

It  is^  however^  physically  possible^  and  indeed  probable^  that  the  cross-spectrum 
has  poles  and  zeros  not  contained  in  either  the  signal  or  the  noise  spectra. 

D.  EVALUATION  OF  THE  G-E.F. 

Having  discussed  the  transient  analogs  associated  with  the  G.E.F.  (Eq  13^)^ 
the  measure  itself  will  now  be  examined-  Its  compatible  criterion  (Eq  153) 


minimum  E^(t) 


was  originally  considered  by  Aigrain  and  Williams  (Ref.  5^)  for  the  optimization 
of  amplifiers  subject  to  step  inputs.  Schultz  (Ref-  48)  and  Schultz  and  Rideout 
(Ref.  28)  continued  the  investigation  of  E-|;^(t)  and  linked  it  directly  to  the 
G-E.F.  The  first  use  of  the  G-E.F.  for  random  inputs  was  by  Lee  and  Wiesner 
(Ref.  5^)^  further  discussion  being  given  by  Spooner  and  Rideout  (Ref.  50) . 
Minimum  G.E.F.  can  also  be  regarded  as  an  alternate  form  of  the  Wiener  optimum 
predicting  (t]  >  O)  or  lagging  (t]  <  O)  criterion  (see  p.  20  ): 


minimum 


X  T  »  .  * 

predicting 
or  lagging 


T 


[r(t  -  Ti)  -  c(t)]  ( 


dt  (173) 


The  G.E.F.  is  thus  very  similar  to  the  for  random  inputs^  and  Ef  is  closely 

p 

related  to  lE^  for  deterministic  inputs.  Much  of  the  following  section  can  be 
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interpreted  as  a  discussion  of  for  cases  where  the  predicting  or  lagging  time^ 
r\,  equals  a  time  constant  which  can  he  chosen  freely. 

As  has  been  shown  in  Ref.  1  and  5^,  the  lE^  performance  measure  exhibits 
poor  selectivity  for  step  inputs^  i.e.^  the  lE^  of  the  optimum  system  is  only 
a  little  less  than  that  of  a  wide  range  of  off-optimimi  systems.  Fig.  5 
demonstrates  that  the  minimum  criterion  is  equally  unselective  for  a  typical 
random  input.  It  has  been  suggested  that  the  G.E.F.  might  achieve  greater 
selectivity  because  it  compares  the  output  with  the  delayed  inputs  and  the 
system  is  therefore  not  penalized  for  ixnavoidable  initial  errors.  However,  the 
principal  conclusions  of  this  section  are 

1  .  the  G.E.F.  is  actually  little  more  selective  than  the  mean 
square  error  criterion 

2.  difficulties  frequently  occur  in  selecting  the  value  of  t 
to  be  used 

3-  the  G.E.F.  does  have  some  merit  in  that  it  possesses  com¬ 
patible  forms  for  both  stationary  random  and  deterministic 
inputs,  and  largely  because  of  this  advantage,  studies  of 
modified  forms  of  G.E.F.  may  be  worthwhile 

To  use  the  generalized  error  function,  it  must  be  put  into  concrete  form  by 
suitable  choice  of  t.  This  problem  will  now  be  discussed. 

Selection  of  t 


This  choice  could  be  related  to  some  of  the  parameters  of  the  system,  but 
perhaps  the  simplest  alternative  is  to  make  the  time  delay,  t,  equal  to  the 
delay  time,  t^^^. 

c('td)  =  ^  c(oo)  (174) 

where  c(oo)  is  the  steady  state  value  of  the 
system  step  response,  and  is  defined  as  the 
time  to  achieve  50  percent  of  the  final  value 
of  the  step  ( indie ial)  response 


This  choice  also  minimizes  the  E-f-(T)  for  a  step  input,  as  will  now  be  shown. 
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Et('r) 


roo  - 

=  I  -  t)  -  c(t)]  dt,  for  r(t)  =  u(t) 

Jo 


(175) 


-  [c(t)  -  r(t)]j  dt 


(176) 


=  f  [^(■t)]^<it  +  f  [r(oo)J^dt  -  2[r(co)]r  e(t)dt  (177) 

Jo  Jo  Jo 


Differentiating  with  respect  to  t. 


dE^(T) 

dx 


[r(cD)]^  -  2[r(oo)]e(T) 


.*•  For  E-j;(t)  to  be  a  minimum^ 


(178) 


e(T^)  =  I  r(co)  (179) 

Et('^)  therefore  has  a  minimum  when  t  is  equal  to  t^^  the  delay  time.  (in  Ref.  28 
this  fact  is  demonstrated  by  use  of  a  rather  cumbersome  geometric  argument.) 

Note  that  the  determination  of  t  =  t^i  is  generally  impossible  by  analytic  means  ^ 
since  t^^^  is  the  solution  of  a  transcendental  equation^  e-g*;  for  a  third-order 
response^  Eq  1 79  becomes 

Ae  ^  +  Be  ^  sin  (pt^  +1)^)  ~  ^  (18O) 

T  must  therefore  be  chosen  by  solving  Eq  1  80  approximately^  or  by  using  the 
standard  approximations  given  in  Ref.  1  . 

The  above  discussion  has  implications  for  random  as  veil  as  step  inputs. 

Using  the  transient  analog  concept^  the  G.E.F.  for  any  random  input  can  be 
minimized  by  choosing  x  to  minimize  Et  for  the  analogous  transient  input.  No 
practical  random  input  yields  a  step  for  its  transient  analogy  because^  as 
shown  in  Eql^i-^^  this  would  imply  infinite  input  power.  However^  if  the  input 
bandwidth  significantly  exceeds  the  system  bandwidth^  the  analogous  transient 
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input  vill  have  time  constants  much  greater  than  those  of  the  system^  and  it 
will  often  he  possible  to  approximate  the  actual  transient  analog  signal  by  a 
step^  as  far  as  the  choice  of  r  is  concerned^  i*e.^  t  =  ('td)step  input^ 
minimum  G.E.F.  For  input  bandwidths  less  than  the  system  bandwidth,  this  arti¬ 
fice  will  fail  and  the  selection  of  t  for  minimum  G-E.F.  must  be  made  on  a 
trial-and-error  basis.  Fig.  29  illustrates  the  generalized  error  function  for 
a  second-order  system. 


H(s) 


_ 1 _ 

s2  +  2^s  +  1 


(181) 


having  an  input  signal  power  spectrum  =  1  /(l  +  a^Tf)  with  zero  noise 

and  T-i  =  1  .  In  this  case  the  value  of  t  at  which  the  minimum  G.E.F.  occurs  is 
hardly  affected  by  but  there  does  not  appear  to  be  any  reason  why  this 
should  generally  be  true.  (Note  that  although  Ref.  30  contains  several  mis¬ 
prints,  the  data  graphed  in  Fig.  29  have  been  checked  and  found  to  be  correct.) 
Even  for  this  simple  case,  the  evaluation  of  the  G.E.F.  is  fairly  tedious,  and 
in  fact  a  digital  computer  was  employed  in  Ref.  30.  Figure  29  (Fig.  4  of 
Ref.  30)  shows  that  the  selectivity  of  the  G.E.F.  with  respect  to  ^  is  poor. 

For  T  =  1 .15,  changing  ^  from  O.5  to  O.7  raises  Et  by  only  4  percent,  approxi¬ 
mately.  E-b(T)  is  minimized  by  ^  =  O.5  at  T  =  1.15-  G.E.F.  thus  gives  about  the 
same  validity  and  selectivity  as  the  simpler  criterion,  and  appears  unpromis¬ 
ing  as  a  performance  criterion.  Some  calculations  have  been  performed  for  other 
values  of  T-;  which  do  not  change  this  conclusion.  On  p.  3^3  of  Ref.  3O;  alternate 
forms  of  G.E.F.  are  suggested,  including  one  with  multiple  delays: 


E(t,  a,  ,  T2)  =  r(t)  -  ac(t  +  )  -  (l  -  a)c(t  +  T2)  (l82) 

where  a  is  a  constant,  and  t-]  and  T2 
are  time  constants 


It  is  possible  that  such  a  G.E.F.  employing  miiltiple  delays  might  achieve 
adequate  selectivity.  Some  further  investigation  along  these  lines  might  be 
worthwhile,  particularly  if  digital  computing  aids  are  available.  The  problem 
of  selecting  the  optimum  t  would  then  become  less  acute,  because  E(t)  coiild  be 
swiftly  calculated  for  a  wide  range  of  t. 
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A  Note  on  the  Evaluation  of  G.E^F*  for  an  Integrated  White  Noise  Input 


The  transient  analog  of  an  integrated  white  noise  Input ^  ^  =  ^/a^y  is 

a  step^  R(s)  =  i/s.  As  noted  previously^  for  such  inputs  it  is  difficult  to 
evaluate  the  G*E.F.  using  Eq  1  ^  because  the  integrals  fail  to  converge.  In 

oi*der  to  demonstrate  how  this  disadvantage  can  he  circumvented^  the  G.E.F.  for 
a  second-order  unit  numerator  system  forced  by  an  integrated  white  noise  input 
will  now  be  evaluated. 

The  system  transfer  function  is 


H(s) 


1 

+  2^s  +  1 


(183) 


Evaluating  E-^^  for  a  unit  step  inputs  instead  of  the  G.E.F.  for  the  actual 
integrated  white  noise  inputs 


/•oo  p 

Et('^)  =  I  -  "t)  -  c(t)]  dt 

Jo 


Adding  and  subtracting  r(t)  to  the  kernel  of  the  integrand  overcomes  the  con« 
vergence  difficulty. 


/•oo  p 

^t('^)  =  I  -  c(t)  +  r(t  -  t)  -  r(t)]  dt 

Jo 


(185) 


L 


00 


G^(t)  +  2G(t)[r(t  -  t)  -  r(t)]  +  [r(t  -  t)  -  r(t)3^  dt  (l86) 


rco  rco  p 

Et(T)  =  lE^  ^  I  “  't’)  “  u(t)3dt  +  I  [u(t  -  t)  -  u(t)3  dt 

JO  Jo 


£ 


=  lE^  -  2  /  e(t)dt  +  T 


(187) 

(188) 


where 


lE^ 


=  r°°Kt)]= 

Jo 


'dt 
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But 


€(t)  =  jT  r  2  1 " 

|_s^  +  2^s  +  lj  P 


(189) 


where  P  =  "I  - 

i}r  =  sin”^  p 


;.  E+(t)  =  IE  +  X  -  +  A 


(190) 


where  A  = 


2e 


-  p^)  sin  Pt  +  2^p  cos  Pt] 


The  choice  of  ^  for  minimum  E^('i')  requires  that  the  equation  L^E-{^(t)/S^3 'r=t^  “  ^ 
he  solved.  This  equation  is  transcendental  and  no  analytic  solution  exists. 

The  results  shown  in  Fig.  50  (Fig.  k  of  Ref.  28)  were  obtained  by  digital  compu¬ 
tation.  The  selectivity  is  little  better  than  that  of  Fig.  29-  This  result 
supports  the  general  assertion  made  above  that  the  G.E.F.  (in  its  present  form) 
has  poor  selectivity. 


Figure  JO.  E-b(T)  vs  x  for  a  Second-Order  Unit  Numerator  System 

H(s)  =  -75 - ^ -  with  a  Unit  Step  Input 

s2  +  2^s  +  1 
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CHAPTER  V 


EREQUEHCY -WEIGHTED  MEAN  SQUARE  ERROR 

Ruchkin  (Ref.  52)  has  modified  the  mean  square  error  performance  measure  to 
deal  with  the  case  when  error  power  is  more  ohjectionahle  in  certain  frequency 
bands  than  in  others.  This  modification  consists  of  the  introduction  of  a 
weighting  function  into  the  expression  for  mean  square  error.  Instead  of 


minimum  e 


2  ^ 


=  minimimi 


-L  r, 

2jt  I 

J-OD 


<I)gg(a3)da) 


(191) 


Ruchkin  proposes  the  criterion 


minimum  EMS^ 


.  .  1 

minimum  7^ 


<5gg(a))W(cD)da) 


(192) 


where  W(a)) 


is  the  frequency  weighting  function 
is  the  error  power  spectinm 


Sections  A  and  B  of  this  chapter  paraphrase  Ruchkin’ s  results^  showing  that  the 
optimum  free-form  system  is  simply  related  to  the  Wiener  system  and  establish¬ 
ing  necessary  restrictions  upon  the  choice  of  W(a)).  Section  C  extends  Ruchkin’s 
analysis  to  show  that  the  criterion 


minimum 


(195) 


can  also  be  expressed  as  a  frequency-weighted  mean  square  error.  Section  D 
considers  the  optimization  of  fixed-form  systems.  The  effect  of  the  choice  of 
W(a))  upon  the  selectivity  and  validity  of  the  EMS^  criterion  is  also  investi¬ 
gated  in  Sections  B  and  C. 
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A.  DERIVATION  OF  THE  MINIMUM  EMSy  SYSTEM 

It  will  now  be  demonstrated  that  the  minimum  EMS\^  system  can  be  obtained  in  an 
analogous  fashion  to  the  Wiener  system  derivation  if  W(a))^  the  frequency  weighting 
in  Eq  19^^  has  certain  properties  in  common  with  the  signal  noise  and  input  power 
spectra*  In  the  most  general  case^  the  input  is  the  sum  of  correlated  signal  and 
noise*  Hence ^ 


^rr  “  ^ss  ^  ^sn  ^ns 


(194) 


where 

ss 

<J>3s(“). 

signal  spectrum 

nn 

^nn(«^)^ 

noise  spectrum 

sn 

<I>sn(j“) 

=  cfoss- 

spectrum  between  signal  and  noise 

It  is  assinned  that  and  have  the  us\ial  Hopf -Wiener  factorization 

ss^  un^  X X 

property^  i.e*^  they  are  even  rational  functions  of  co^  and  therefore^ 


$(a))  =  jco)  =  i  ^  (195) 

where  <!>"*"(  joi)  has  poles  and  zeros  only  in  the 
left  half -plane 

<j)"’(jcD)  has  poles  and  zeros  only  in  the 
right  half -plane 

W(od)  is  also  assumed  to  be  Hopf-Wiener  factorable.  Therefore,  EMSy  can  be 
obtained  by  filtering  the  error  signal  with  a  system  having  the  transfer  function 

p 

W^(ja))  (as  shown  in  Fig.  51  )  evaluating  e^. 


Figure  51 .  Block  Diagram  of  System  with  Signal  Plus  Noise  Input 
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From  Fig.  31  it  can  be  seen  that  the  frequency-weighted  mean  square  error  is 


EMSy 


J_ 


2it 


(cD)W(a))da> 


(196) 


where  W((jo) 
and  p  (od) 


W+(ja))w'‘(-ja)) 

the  spectrum  of  the  weighted  error 


The  error  power  spectrum  expressed  in  terms  of  the  signal,  noise,  and  cross 
spectra  is  the  same  as  in  the  Wiener  problem.  From  Ref.  50,  p.  239, 


|2 

-  H  ■'■bs  + 


-  - 


H 


ns 


(197) 


where 


T]  is  the  prediction  time 

H  =  H(oa))  is  a  general  system  transfer 
function 


=  H(-joD) 


The  system  that  minimizes  the  frequency-weighted  mean  square  error  can  be 
found  by  using  the  formula  for  the  Wiener  optimum  linear  system  which  is  (as 
shown  in  Chapter  I) 


$  + 
rr 


(198) 


If,  in  Eq  197,  and  are  replaced  with  W^^^,  W3>^^, 

and  W5>ns,  respectively,  then  the  left-hand  side  becomes  W<iUc  (  =  <I>  ).  With 

4-v,  -  wV 

these  substitutions,  the  problem  of  minimizing  EMS  ,  Eq  1 96,  can  be  regarded  as 

that  of  minimizing  the  mean  square  error  using  the  modified  input  spectrum  W4>yj, 

givgP  aboire.  Inserting  these  modified  spectra  into  Eq  198  yields  the  optimum 

EMS^  linear  system,  as 


Hem%(J“) 


*  +W+ 
^rr” 


ej^(<& 


ss  +‘^ns)W^' 


4)  -u- 

rr’^ 
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+ 


(199) 


which  can  be  simplified  to  the  result  given  by  Ruchkin  (Ref.  52) : 


ss 


+  «&  )W'' 

_ asi _ 


rr 


(200) 


For  W((Jo)  =  1  the  optimiJiii  EMS-^  system  becomes  the  Wiener  system.  Ruchkin  does  not 
consider  the  characteristics  of  the  optimum  EMSy  system  and  proceeds  to  discuss 
the  necessary  restrictions  upon  W(a))  that  are  implicit  in  Eq  200.  However^  it 
requires  only  a  brief  digression  to  show  that  the  optimum  EMS^  system  is  simply 
related  to  the  Wiener  system.  This  will  now  be  proved  before  continuing  to  dis¬ 
cuss  the  limitations  on  W(a3)  .  It  will  be  shown  that  the  optimum  EMSy  system  can 
be  represented  by  two  systems  in  parallel^  one  of  which  is  the  Wiener  system  for 
the  given  signal  and  noise.  For  the  time  being  it  will  be  assumed  that  W(a))  has 
an  equal  number  of  zeros  and  poles.  The  reasons  for  this  assumption  are  made 
clear  in  Section  B. 

Equation  200  can  be  written  as 


[(A  +  B)W+] 


(201 ) 


where  A  =  A(jai)  is  that  part  of  the  partial  fraction 

^  jo^n/^ss  "^^ns^ 

expansion  of  e"^  1 - - - 1  with  only 

\  / 

left- half -plane  poles 

B  =  B(j(D)  is  that  part  of  the  partial  fraction 

„  joTi/^ss  ’^^ns\  ...  _ 

expansion  of  e"^  M - - jwith  only 

^  ^rr  ' 

right-half -plane  poles 

Equation  201  can  be  rewritten  as 


[aw+]^  +  [bw+]^ 


(202) 


Since  AW’*'  has  only  left-half -plane  terms,  202  becomes 


108 


(203) 


rr 


and  this  can  be  written  in  terms  of  the  Wiener  system  as 


where 


% 


Hv 


A 


<&  + 
rr 


the  Wiener  system 


the  system  in  parallel 
with 


A  and  B  are  as  in  Eq  201 


(204) 


Thus,  can  be  expressed  as  two  systems  in  parallel,  one  of  which  is  simply 

the  Wiener  system- 

Example  of  Optimum  Frequency-Weighted  Mean  Square  Error  System 

As  an  example  'Xb  illustrate  the  foregoing^  consider  the  following  uncorre¬ 
lated  signal  and  noise  input  spectra: 


1+0^  1  -  s^ 


s=ja3 


(1  +  s)(l  -  s) 


<I>  = 

nn 


rr  ss  nn 


1  -  s 


+ 


1  +  -  N^s^ 

1  -  s2 


(205) 


And  since  is  Hopf -Wiener  factorable, 

+  _  +  1  +  Ns 

rr  1  +  s 

+  1  -  Ns 

^rr  1  -  s 


(206) 
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H- 


:^3yjg^  is  given  by  Eq  200.  For  t)  =  0, 


H- 


EMS^ 


''^’ssW^' 

w+ 

- 

<h 

_^rr  ^ 

+ 

N(l  +  s) (A  -  s) 

A 

+  s 

rr 

1 

+  s 

where  A 

Vn^  + 

N 

1 

W+ 


(207) 


Expanding  <&  in  partial  fractions  yields 


^SS  _  _ 


+l)\1+s  A-s/ 


(208) 


Therefore^  from  Eq  201^  A(s)  and  B(s)  are 


A(s) 


1 

W(A  +  1  )(1  +  s) 


B(s) 


1 

N(A  +  1)(A  -  s) 


(209) 


Substituting  Eq  206  and  209  into  Eq  205  for  Hej4S^(s)  yields 


HEMSy(s) 


_ 1 _ 

K^(A  +  1)(A  +  s) 


+ 


W+ 

1J(A  +  1 )  (A  - 


(210) 


where  the  first  term  is  the  Wiener  system  for  the  given  signal  and  noise. 
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B.  RESTRICTIONS  UPON  THE  CHOICE  OF  THE  FEIEQUENCY  WEIGHTING  FUNCTION 


The  restrictions  upon  the  choice  of  the  frequency  weighting  function  will 
now  be  illustrated  by  continuing  the  above  example.  Assume  the  following  form 
for  W*'(s) : 


W+Cs)  = 


1  +  as 

1  +  ps 


(211) 


Inserting  this  into  the  second  term  of  Eq  210  yields  (using  Eq  20^) 


Hx(s)  = 


[n(a 


1  +  OS 
+  1)(A  -  s)(l  +  ps) 


0  -f) 


N(A  +  1)(A  +  i)(l  +  ps) 


E(A  +  s)  / 

1  +  as\ 

N(A  +  s)  / 1  +  as\ 

1  +  s  \ 

1  +  ps/ 

1  +  s  \1  +  ps/ 

(212) 


which  can  he  simplified  to 


(1  -  ^)(l  +  s) 

Hx(s)  =  - 2 -  (213) 

N^(A  +  1 ) (A  +  1) (A  +  s) (l  +  as) 

P 

Ruchkin  states  that  W^(s)  must  have  at  least  as  many  zeros  as  poles.  A  simple 
demonstration  of  this  is  obtained  by  letting  a—^O  in  Eq  21 1  and  inspecting  the 
final  result.  From  Eq  215^  H^Cs)  then  becomes 

Hx(s)  =  - -  (214) 

n2(A  +  1)(A  +  i)(A  +  s) 

P 

This  system  has  a  finite  gain  at  all  frequencies  and^  since  the  noise  part  of 
the  input  has  infinite  bandwidth,  the  output  power  will  be  infinite;  clearly  an 
undesirable  result.  It  can  be  shown  that  if  W^(s)  had  been  a  ratio  of  second- 
order  polynomials  in  s  and  the  above  procedure  followed,  then  Hx(s)  would  have 
one  more  zero  than  pole.  Therefore,  it  would  be  physically  iinrealizable  in 
addition  to  having  infinite  output  power.  Hence,  the  weighting  function  must 
have  at  least  as  many  zeros  as  poles.  To  see  that  it  may  have  more  zeros  than 
poles,  allow  p— ►0  in  Hx(s),  Eq  215,  which  yields 


in 


H„(s)  . 


N"^(A  +  1  )(A  +  s)(l  +  as) 


and  therefore  Hej^^(s),  Eq  20h,  becomes 


HemS^(s)  = 


a(l  +  s) 


N^(A  +  1)(A  +  s)  N"^(A  +  1)(A  +  s)(l  +  as) 


which  could  he  combined  to  yield 


(215) 


(216) 


HeM%(s) 


_ 1  -  g _ 

n2(A  +  1)(A  +  s)(1  +  as) 


(217) 


C.  PEEQUENCY  WEIGHTING  AS  A  MEANS  OF  REPRESENTING  A  GENERAL 
QUADRATIC  CRITERION  INVOLVING  DERIVATIVES  OF  ERROR 


The  criterion 


minimum  .  ) 


(218) 


has  been  suggested  (in  a  more  general  quadratic  form)  by  Bellman  (Ref.  55)  and  by 
Kalman  and  Koepcke  (Ref.  56)  for  dynamic  programming  optimization  techniques. 
This  is  easily  interpreted  as  a  frequency-weighted  fom 


EMSy 


4 1 


00 


(of 


00'^ 


2  2 


2  k 

g^ar 


W(co) 


)^ge(a3)d<n 


(21  9) 


where  use  has  been  made  of  the  following  relationships: 


J_ 

2jt 


^-(05)100 


1  ■ 

J-00 

^  f  (D^d)^g((n)da) 

J-00 


(220) 
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The  fomula  for  the  optimiom  linear  system^  given  hy  Eq  20k,  still  applies. 
An  example  of  optimization  vith  this  criterion  is  given  in  the  previous  section 
(see  Eq  215^  2l6^  and  217)  where  it  can  be  seen  that  the  weighting  function 
Eq  2n  for  p— is 


W+(s)  = 


lim  1  +  as 

0  1  +  ps 


=  1  +  as 


therefore 


W(co)  =  1  + 


(221) 

(222) 


and  therefore  in  this  case 


EMS^  =  +  a^€^ 


(223) 


The  validity^  selectivity^  and  ease  of  application  of  this  criterion  will  be 

tested  in  Section  where  it  will  be  applied  to  a  second-order  unit  numerator 

system  H(s)  =  .  where  t  is  the  variable  parameter, 

s^  +  2^s  +1 

D.  FIXED-FORM  (PHILLIPS -TYPE)  OPTIMIZATION  OF  EMSy 

This  section  derives  the  optimum  fixed- form  (Phillips)  system  of  the  form 

'j 

H(s)  =  2^2^ — different  frequency  weighting  functions: 


Wa(a))  = 


Wb(co)  = 


lim  1  +  as 


p-*-0  I  1  +  ps 


+  2^ 1 s  +  1 


( 1  +  a^o^) 


s'-  +  2^2^  +  1 


(224) 

(225) 


The  reason  for  the  limiting  process  in  Wa(ai)  will  become  clear  in  the  discussion. 
It  will  be  shown  that  neither  nor  yields  a  valid  and  selective  criterion. 

As  demonstrated  in  the  previous  section^  inserting  the  weighting  function 
Wa(^h)  into  EMS^^  yields  the  simplest  form  of  Bellman's  performance  measure: 

EMSy  =  +  a^e^  (226) 
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EMS^  (Eq  1 92)  could  be  evaluated  on  an  analog  computer  using  the  circuit  in 
Fig.  51  where  W'*"(ja})  is 

=  1  +  as  (227) 


But  even  with  operational  amplifiers  this  is  not  strictly  realizable,  since  any 
differentiating  circuit  will  fall  off  at  very  high  frequencies.  Therefore^  the 
use  of 


W+(ja)) 


1  +  as 

1  +  ps 


(228) 


where  p  is  much  smaller  than  any  of  the  time  constants  of  the  error  signal  would 
he  a  more  realistic  and  practical  system.  It  will  he  shown  that  the  optimiza- 
tlon  of  H(s)  .  -n  an  integrated  white  spectrimi  desired  input 

signal  (zero  noise)  is  essentially  independent  of  p  if  p  is  sufficiently  small. 

The  terms  comprising  4>gg(a5)  (Eq  197)  are  therefore 


«>< 


ss 


J_ 

a? 


(229) 


■^nn 


<J)  =0 

^ns  ^ 


H(s)  = 


n  = 


1 


s2  +2^8+1 
0 


and  therefore  the  integrand  of  EMS^  becomes 


/  s  +  2^  \ 

/l  +  as\  1 

\s2  +  2^s  +  1/ 

\l  +  Psj  1 

2 

S=jCD 


(250) 


Evaluating  EMS^j  using  the  tables  in  Ref.  6  yields 


EM%  = 


a^(2i;  +  p)  +  [(g^g  +  1)^  -  Halp  +  4{:^(2t:p  +  1)P 

2p[(2^  +  p)(2p^  +  1)  -  p| 


(231) 


Dividing  numerator  and  denominator  dy  p  and  sx]25)lifying^  this  becomes 


EMSy  = 


+  1 )  (a^  +  1 )  + 

4^(1  +  2^P  +  p2) 


(232) 


For  small  P  the  last  term  in  the  numerator  is  negligible  and  Eq  232  becomes  approx¬ 
imately 

EM%  4  +^)(a2  +  1)  +  / — ^ (235) 

\2p(l  +  2^p  +  p^)/ 

Expanding  the  last  term  in  a  Taylor  series  about  p  =  0  yields 


EMSy  =  (^  +  ]^)  (ct.2  +  1  )  +  ^  {l  -  2^p  +  p2(4^2  _  -j  )  .  .  .| 


2P 


(234) 


Expanding  Eq  234  yields 


EMSw  =  (^  +  ■i^)(a.2  +  1)  +  ^  -  ^a2  +  a2(4^2  _  i  )p  _ 


2P 


(235) 


and  for  small  p  the  frequency-weighted  mean  square  error  finally  becomes 


EMS^  = 


1  + 

■TT" 


(236) 


The  value  of  ^  that  minimizes  Eiy%  is  found  by  differentiating  Eq  236  with 
respect  to 


dEMS^  ,  1  +  a2 

^  "  4^2  - 


(237) 


and  solving  for  ^  yields 


Vi  - 

2 


(238) 


For  this  criterion  to  be  valid  (^  =  O.7)  requires  an  a  of  about  one.  Eq  236 
is  graphed  in  Fig.  32  for  a  =  1 ,  p  =  1 .  Fig.  32  shows  that  the  selectivity  of 
this  EWS^j  criterion  is  no  better  than  that  of  ^ 
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Performance  Measure 


2.2 


Figure  52. 


Comparison  of  and  +  g2  for  H(s)  =  — - 

s2  +  2^s  +  1 

vith  an  Integrated  White  Spectrum  Input  $^^(0))  = 


n6 


As  an  example  of  optimization  with  the  second  weighting  function,  Eg  225, 


W^(cd)  = 


s^  +  2^^  s  +  1 


+  2^2^  +  I 


(239) 


S=JCJD 


the  same  system,  H(s),  and  input  of  the  previous  example  will  be  used.  For 
=  1  and  ^2  «  ^  J  weighting  function  will  tend  to  accentuate  the  mid- 

freguency  components  of  which  depend  mainly  on  ^ .  One  would  therefore 
expect  the  criterion  to  he  highly  selective  for  these  values  of  and  ^2" 
Therefore, 


<J)  W  = 
ee 


/  S  +  2(;  ' 

v/s2  +  2^ts  +  n 

\s2  +  2^s  +  1, 

/\s2  +  2^2^  +  ^  ' 

S=JCD 


(240) 


Using  the  table  of  Ref.  6,  EMSy  becomes 


♦  1)(U2  + 


(241) 


This  is  plotted  in  Fig.  33  as  a  function  of  ^  for  various  values  of  and 
^2"  In  addition,  plots  of  ITAE,  ITE^,  and  IT^E^  for  a  step  input  to  the  same 
system  are  reproduced  from  Ref.  1  for  selectivity  comparisons.  The  analytic 
fomns  of  these  performance  measures  are  rational  functions  of  ^  and  therefore 
they  have  been  plotted  "Bode  fashion,"  taking  advantage  of  their  asymptotic 
character  (except  for  ITAE  which  contains  transcendental  terms) .  It  must  be 
remembered  though  that  only  positive  real  values  of  ^  are  considered  in  the 
"Bode  plots . " 

The  eS  can  be  obtained  from  EMSy  when  graphed  in  Fig.  33  as 

curve  •  The  selectivity  is  poor.  The  steepest  possible  asymptotes  obtain¬ 
able  using  the  weighting  function  of  Eg  239  actual  EMS^  are  shown  as 

curve  .  The  selectivity  is  little  better  than  that  of  the  mean  sguare  error. 
In  comparison,  "good"  selectivity  is  indicated  by  curves  ,  (J) ,  and  (^  for 
ITAE,  IT^E^,  and  ITE^,  respectively. 
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(4£^+ l)(«2+C|^) 

4££g(£  +  {2) 

0  C2=^l 

®  Sg'O.! 

£|  =0.224 


®  ITAE (formula  given  in  Ref  l) 


- @  IT^E2=2£®-f+^ 

- (D  ITE^  =  £2  +  ^ 


Figure  35*  Selectivity  Comparisons  of  EMS^^  ITAE,  ITE^,  and  IT^E^ 


CHAPTER  VI 


A  NOTE  ON  COMBINED  AND  CONSTRAINED  CRITERIA,  AND  CONCLUSIONS 

The  term  "combined  criteria"  is  used  to  denote  performance  criteria  that  are 
functions  of  more  than  one  dependent  variable,  e.g.. 

The  generalized 
error  function, 

Glover's  criterion,  minimum  P.E.  =  minimum  f(€,  c,  6) 

Combined  criteria  can  sometimes  be  expressed  as  frequency-weighted  forms,  e.g., 
minimum  +  /cg^  is  equivalent  to  minimum  ^  J  ^  W(cD)$^^(a))dcD^  where  W(u))  is  a 
frequency-weighting  factor  incorporating  a  constant  K  which  expresses  the  rela¬ 
tive  importance  of  error  and  time  rate  of  change  of  error  quantities  as  measures 
of  the  dynamic  performance  of  a  given  system. 

In  order  to  put  combined  criteria  into  concrete  usable  foms^  the  linking 
constants  or  equivalent  frequency-weighting  functions  must  be  chosen  to  reflect 
system  requirements .  The  present  generalized  study  can  offer  little  advice  on 
the  choice  of  these  weighting  parameters;  in  Chapters  IV^  and  V  we  have 

attempted  to  illustrate  the  consequences  that  follow  once  certain  choices  of 
these  parameters  are  made.  The  reader  must  decide  for  himself  whether  these 
choices  are  appropriate  for  the  particular  system  that  he  is  studying. 

Combined  criteria  are  frequently  expressed  in  the  form  "minimum  f(s)"  with  a 
constraint  on  some  other  parameter^  such  as  bandwidth  or  peak  power.  Such  con¬ 
strained  criteria  have  been  extensively  studied  by  Westcott  (Ref.  11 ),  Newton^ 
Gould,  and  Kaiser  (Ref.  6),  Chang  (Ref.  10),  Hung  (Ref.  9),  and  others.  Westcott 
noted  that  constrained  criteria  could  be  divided  into  two  classes: 

Class  1  :  A  condition  is  imposed  upon  the  system  transfer 
function  either  at  a  particular  frequency  or  as 
a  trend  with  frequency. 

Class  2:  The  constraint  is  expressible  as  an  integral 

over  the  range  of  the  function  being  varied  and 
is  required  to  have  a  constant  value  or  to  be 
less  than  some  specified  value,  e.g.,  limiting 
of  total  energy. 

Reference  1 1  and  most  subsequent  analyses  of  random  input  performance  criteria 


minimum  G.E.F.  =  f(€,  x) 
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deal  with  Class  2  constraints.  The  difficulty  in  selecting  reasonable  values  of 
the  weighting  parameters  to  assess  combined  criteria  becomes  aggravated  when 
constraints  are  introduced.  Mathematical  procedures  required  to  obtain  the 
optimimi  system  have  been  detailed  in  Ref.  6  and  10.  In  the  present  report  only 
a  brief  note  will  be  given  to  indicate  the  techniques  that  are  available;  details 
of  the  con^utational  procedures  can  be  found  in  the  references  cited. 

Newton^  Gould,  and  Kaiser  consider  constrained  criteria  of  the  form 
"minim-urn  "  with  a  constraint  upon  the  rms  input  to  a  supposedly  nonlinear 

component  of  the  system.  The  object  is  to  optimize  the  system  subject  to  the 
restriction  that  the  nonlinear  element  must  operate  in  the  linear  (unsaturated) 
part  of  its  range.  The  analytical  procedure  adopted  consists  of  the  replacement 
of  the  nonlinear  element  by  a  substitute  linear  component  having  identical  char¬ 
acteristics  to  those  of  the  actual  component  over  the  linear  part  of  its  operating 
range.  The  rms  input  to  the  substituted  linear  element  is  then  constrained  to  be 
less  than  that  maximum  input  amplitude  for  which  the  actual  system  remains  linear. 
It  is  shown  in  Ref.  6  that  this  procedure  is  equivalent  to  minimizing  the  proba¬ 
bility  of  saturation,  provided  that  that  probability  is  small.  Lagrange 
multipliers  are  used  to  incorporate  the  constraint  into  the  minimization  procedure. 

Related  techniques  are  used  in  Ref.  6,  p.  21 5^  to  minimize  mean  square  error 
subject  to  a  constraint  on  bandwidth,  althoi:igh  the  term  "bandwidth"  is  used  only 
in  a  rather  general  sense  as  "the  frequency  over  which  the  system  will  have  an 
output  approximately  equal  to  the  desired  output."  Such  a  definition  is  too  loose 
to  be  incorporated  directly  into  standard  variational  calculus  or  other  minimiza¬ 
tion  procedures,  and  the  bandwidth  is  therefore  calibrated  in  terms  of  the  mean 
square  noise  transmitted  by  a  specified  standard  filter.  The  mean  square  error 
is  then  minimized  with  this  "calibrated  constraint"  being  taken  in  accoiint  by 
means  of  Lagrange  multipliers.  Ref.  6  presents  examples  of  this  optimization 
procedure  for  deterministic  inputs  only;  however,  as  noted  in  Ref.  9^  the  modifi¬ 
cations  required  to  deal  with  stationary  random  inputs  are  easily  made.  In 
Ref.  9  the  bandwidth  of  the  input  to  a  specified  component  is  constrained  to 
avoid  exciting  undesirable  high-frequency  modes  associated  with  that  component. 

In  flight  control  system  design  this  technique  could  be  useful  in  minimizing  mean 
square  error  without  exciting  aeroelastic  or  autopilot  modes. 
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The  computational  labor  involved  in  the  analytic  minimization  of  constrained 
performance  measures  is  often  formidable.  This  point  is  noted  on  p.  246  of 
Ref.  Gy  where  it  is  suggested  that  the  analytic  minimization  procedure  should  be 
used  mainly  as  a  guide  for  trial-and-error  optimization  procedures,  "pilot” 
analytic  minimization  using  simplified  systems  and  inputs  being  made  to  approxi¬ 
mately  determine  the  lowest  attainable  value  of  the  particular  performance 
measure  being  considered.  Some  progress  toward  shortening  this  process  has  been 
made  by  Chang-  His  "root  square  locus"  method  (Ref.  10)  goes  some  way  toward 
combining  the  precision  of  analytical  optimization  techniques  with  the  ease  of 
iteration  afforded  by  graphical  procedures. 
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CONCLUSIONS 


1 .  Performance  criteria  for  linear  constant -coefficient  systems  with  random 
inputs  have  been  investigated^  with  particular  reference  to  flight  control 
systems.  The  application  of  performance  measures  has  been  facilitated  by  substi¬ 
tuting  for  the  actual  flight  control  system  an  "equivalent"  low-order  linearized 
system  having  similar  dynamic  characteristics.  This  equivalent  system  was  con¬ 
structed  by  dividing  the  actual  system  transfer  function  into  regions  of  interest 
defined  by 
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(c)  |G(jCD)|  =  1 


The  fom  of 


in  the  last  region  defines  the  dominant  modes  of  the 


closed- loop  system  response^  and  can  usually  be  closely  approximated  by  a  system 
of  first,  second,  or  third  order. 


2.  For  stationary  random  inputs,  many  criteria  are  equivalent  to  minimum  e^. 
This  criterion  is  relatively  easy  to  apply  using  "pencil  and  paper"  techniques 
if  use  is  made  of  the  equivalent  system  concept.  It  possesses  a  compatible 
deterministic  criterion  (minimum^  e^dt);  hence,  operations  upon  random  quanti¬ 
ties  required  to  compute  ^  can  be  replaced  by  operations  upon  their  more  easily 
visvtalized  transient  analogs.  Its  principal  disadvantages  are  that  it  yields 
rather  lightly  damped  systems  and  is  unselective,  i.e.,  the  of  the  optimum 
system  is  only  slightly  less  than  that  of  a  wide  range  of  off -optimum  systems. 


5.  No  criterion  has  been  found  that  has  the  advantages  of  minimum  with¬ 
out  comparable  disadvantages. 

Of  those  criteria  that  did  not  reduce  to  minimum  the  following 
deserve  further  investigation: 
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a.  Exceedance  criteria  (for  stationary  Gaussian  inputs) : 
Exceedances  are  relatively  easy  to  calculate^  but^  in 
most  of  the  examples  studied^  gave  very  heavily  dairped 
systems.  The  implications  on  dynamic  performance  of 
the  growing  use  of  exceedances  as  fatigue  criteria 
requires  consideration. 

b.  Frequency- weighted  criteria:  These  may  be  suitable 
where  the  frequency-weighting  can  be  specified  so  that 
it  directly  reflects  performance  requirements.  Further 
study  is  required  to  determine  whether  this  is  feasible 
for  a  class  of  flight  control  systems  sxifficiently 
broad  to  be  of  general  interest. 

5.  Zaborszky  and  Diesel’s  probabilistic  square  error  criterion  is  suitable 
for  nonstationary  situations  where  the  weighting  function  describing  the  varia¬ 
tion  of  error  importance  with  time  can  be  specified. 

6.  A  wide  variety  of  criteria  have  been  surveyed  and  many  new  results  and 
techniques  have  been  produced.  These  include: 

a.  Proof  that  for  stationary  inputs  Muiphy  and  Bold’s 
time-weighted  square  error  criterion  reduces  to 
minimum 

b.  Simplified  procedures  for  evaluating  the  probabilistic 
square  error  measure. 

c.  Generalized  definition  of  the  concept  of  compatible 
criteria  and  exploitation  of  this  concept  to  obtain 
transient  analogs  for  both  uncorrelated  and  corre¬ 
lated  signal  and  noise. 

d.  A  technique  for  applying  Glover’s  amplitude-weighted 
criterion  to  fixed-form  systems. 
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